Microscopic imaging system and method, device, and readable storage medium

Abstract

A microscopic imaging system and method, a device, and a readable storage medium. The system comprises: a light source (1), which is arranged at the start end of an illumination path and provides illumination for a stage (2) of a microscope (110); a filter assembly (5), which alternately provides a red light filter, a green light filter and a blue light filter to the illumination path or an imaging path; a monochrome camera (4) which is arranged at the tail end of the imaging path and configured to generate R / G / B channel monochrome images corresponding to the red light filter, the green light filter and the blue light filter; and a processor (6), which is electrically connected to the monochrome camera (4), and is configured to process the R / G / B channel monochrome images by means of an image fusion algorithm to obtain a color image. Without significantly changing the original structure of the microscope (110) system, and by only connecting the filter assembly (5) to the existing microscope (110) system as an accessory, color imaging and monochrome imaging can be achieved by using one monochrome camera (4) in one microscope (110) without reducing the imaging quality and stability.

Description

Microscopic imaging system, imaging method, device and readable storage medium Technical Field

[0001] The present application relates to the field of microscope imaging technology, and in particular to a microscope imaging system, imaging method, device and readable storage medium. Background Art

[0002] Microscopes usually need to have both color and monochrome imaging capabilities to adapt to different application requirements. For example, in pathology research and analysis applications, users usually require color imaging for accurate diagnosis; while in applications where imaging is performed in weak signal environments such as fluorescence, monochrome imaging is required to achieve high imaging sensitivity.

[0003] Currently, there are microscopes with color imaging and monochrome imaging functions. One feasible solution is shown in Figure 6: a black and white camera 4 and a color camera 6 are connected to the microscope system, and a beam splitter 34 is used to guide the imaging light path into the black and white camera 4 and the color camera 6 respectively. Specifically, the light source 1, the stage 2, the objective lens 31, the beam splitter 34, the upper tube lens 32 in Figure 6, and the black and white camera 4 form the light path for monochrome imaging, while the light source 1, the stage 2, the objective lens 31, the beam splitter 34, the reflector 33, the lower tube lens 32 in Figure 6, and the color camera 6 form the light path for color imaging.

[0004] Another feasible solution is shown in Figures 7 and 8: the black and white camera 4 and the color camera 6 are also connected to the microscope system, and a movable reflection module is used to selectively guide the imaging light path into the black and white camera 4 or the color camera 6. Specifically, when the reflection module moves to the left of the reflection mirror 33 in Figure 7 and is aligned with the color camera 6, the microscope forms a color image; when the reflection module moves to the right of the reflection mirror 33 in Figure 8 and is aligned with the black and white camera 4, the microscope forms a monochrome image.

[0005] The disclosure of the above background technology content is only used to assist in understanding the inventive concept and technical solution of the present invention. It does not necessarily belong to the prior art of this patent application, nor does it necessarily provide technical guidance. In the absence of clear evidence that the above content has been disclosed before the filing date of this patent application, the above background technology should not be used to evaluate the novelty and creativity of this application. Summary of the Invention

[0006] The object of the present invention is to provide a microscopic imaging system which can realize color imaging when only a black and white camera is provided.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0008] A microscope imaging system, comprising:

[0009] a light source configured to provide illumination to a stage of the microscope;

[0010] a filter assembly configured to alternately provide a red light filter, a green light filter, and a blue light filter to an illumination path or an imaging path of the microscope;

[0011] a black and white camera, disposed at an end of the imaging path, configured to generate an R-channel monochrome image, a G-channel monochrome image, and a B-channel monochrome image corresponding to the red light filter, the green light filter, and the blue light filter; and

[0012] A processor module is electrically connected to the black and white camera, and is configured to process the R channel monochrome image, the G channel monochrome image, and the B channel monochrome image through an image fusion algorithm to obtain a color image.

[0013] Furthermore, based on any one of the technical solutions or a combination of multiple technical solutions described above, the duration for which the filter assembly provides the same filter to the illumination path or the imaging path is greater than or equal to a single exposure time of the black and white camera.

[0014] Further, based on any one of the technical solutions or a combination of multiple technical solutions described above, the filter assembly includes a turntable and a driving mechanism, the turntable is configured with at least three circumferentially arranged holes, and a red light filter, a green light filter and a blue light filter are respectively set in the holes; the driving mechanism is configured to drive the turntable to rotate so that the three holes reach the illumination path or the imaging path in turn.

[0015] Furthermore, based on any one of the technical solutions or a combination of multiple technical solutions described above, the turntable is integrated with an encoding disk and a sensor, and the processor module is configured to be electrically connected to the sensor to obtain the real-time position of the turntable.

[0016] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, the number of hole positions arranged circumferentially on the turntable is four, and the fourth hole position is an empty hole position;

[0017] The black and white camera is further configured to generate a black and white image when the empty hole position arrives on the illumination path or the imaging path.

[0018] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, the microscope imaging system further includes an objective lens and a tube lens, which are sequentially arranged on the optical path between the light source and the black and white camera;

[0019] The filter assembly is arranged between the objective lens and the tube lens, and the distance between the filter assembly and the objective lens is smaller than the distance between the filter assembly and the tube lens.

[0020] Furthermore, based on any one of the technical solutions or a combination of multiple technical solutions described above, the central wavelength of the red light filter is between 600 and 700 nm; and / or, the central wavelength of the green light filter is between 500 and 600 nm; and / or, the central wavelength of the blue light filter is between 400 and 500 nm.

[0021] Furthermore, based on any one of the technical solutions or a combination of multiple technical solutions described above, the red light filter, the green light filter and the blue light filter are all strip-shaped pieces.

[0022] Furthermore, based on any one of the technical solutions or a combination of multiple technical solutions described above, the widths of the red light filter, the green light filter and the blue light filter are greater than the optical path width of the illumination path or the imaging path, and the edges of the filters in the longitudinal direction are arc-shaped.

[0023] Furthermore, according to any one of the above technical solutions or a combination of multiple technical solutions, each R channel monochrome image, G channel monochrome image, and B channel monochrome image is sequentially assigned a serial number according to the order of imaging;

[0024] The processor module reads the serial number of the image, sequentially fuses three monochrome images with adjacent serial numbers to obtain a frame of color image, and generates a color video with multiple frames of color images.

[0025] Furthermore, based on any one of the technical solutions or a combination of multiple technical solutions described above, the color video is a real-time color video, and its color video frame rate matches the frame rate of the R channel monochrome image, the G channel monochrome image, and the B channel monochrome image.

[0026] Furthermore, based on any one of the technical solutions or a combination of multiple technical solutions described above, the light source is a white light source.

[0027] According to another aspect of the present invention, there is provided a microscope imaging method comprising the following steps:

[0028] Turn on the light source so that it illuminates the microscope stage;

[0029] driving the filter assembly to rotate so that the red light filter, the green light filter, and the blue light filter of the filter assembly are alternately positioned in the optical path of the microscope;

[0030] Controlling a black and white camera to expose the red light filter, the green light filter, and the blue light filter when they arrive in the light path, respectively, to obtain an R channel monochrome image, a G channel monochrome image, and a B channel monochrome image;

[0031] The R channel monochrome image, the G channel monochrome image, and the B channel monochrome image are processed by an image fusion algorithm to obtain a color image.

[0032] Furthermore, based on any one of the aforementioned technical solutions or a combination of multiple technical solutions, serial numbers are assigned to the R channel monochrome image, the G channel monochrome image, and the B channel monochrome image, and video frames of the color video are generated according to the serial numbers.

[0033] Furthermore, based on any one of the aforementioned technical solutions or a combination of multiple technical solutions, the duration of each of the red light filter, the green light filter or the blue light filter in the optical path is greater than or equal to the single exposure time of the black and white camera.

[0034] Furthermore, based on any one of the aforementioned technical solutions or a combination of multiple technical solutions, the black and white camera is controlled to perform exposure when an empty hole position of the filter assembly reaches the light path to generate a black and white image.

[0035] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, the wavelengths and transmittances of the red light filter, the green light filter, and the blue light filter are set based on the spectrum of the light source and the quantum efficiency of the black and white camera so that the color image achieves a preset color reproduction degree;

[0036] Alternatively, based on the spectrum of the light source, the quantum efficiency of the black and white camera, and the spectral transmittance of the optical elements in the microscope, the wavelengths and transmittances of the red light filter, the green light filter, and the blue light filter are set so that the color image achieves a preset color reproduction degree.

[0037] According to another aspect of the present invention, an electronic device is provided, comprising a processor and a memory, wherein the memory is used to store program instructions, and the processor is configured to run the program instructions, characterized in that the program instructions are run to perform the steps of the method described above.

[0038] According to another aspect of the present invention, a computer-readable storage medium is provided for storing program instructions, wherein the program instructions are configured to be called to execute the steps of the above-mentioned method.

[0039] According to another aspect of the present invention, a computer program product is provided, comprising a readable and stored computer program, wherein the computer program comprises program instructions, and is characterized in that when the program instructions are executed on a computer device, the computer device executes the steps of the above-mentioned method.

[0040] The beneficial effects brought about by the technical solution provided by the present invention are as follows:

[0041] a. Color imaging can be achieved using only a black-and-white camera and a rotatable filter assembly. This makes it easy to integrate into existing microscope systems. Apart from the filters, the mechanical turntable for mounting the filters, and the motor that drives the turntable, there is virtually no hardware modification cost.

[0042] b. By providing empty holes in the filter assembly, color imaging can be achieved while also taking into account monochrome imaging applications;

[0043] c. Adding a rotatable filter assembly does not affect the alignment between the light source, objective lens, tube lens, and camera, nor does it increase the complexity of the system;

[0044] d. In both monochrome and color imaging applications, 100% of the light is utilized to achieve imaging, resulting in high imaging sensitivity. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technical solutions, the drawings required for use in the description of the embodiments or the conventional technical solutions will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] FIG1 is a schematic structural diagram of a microscopic imaging system provided by an exemplary embodiment of the present invention, wherein a filter assembly is located between an objective lens and a reflective mirror of a microscope;

[0047] FIG2 is a schematic diagram of the structure of a microscopic imaging system provided by an exemplary embodiment of the present invention, wherein the filter assembly is located between the reflector and the tube lens of the microscope;

[0048] FIG3 is a schematic diagram of the structure of a microscopic imaging system provided by an exemplary embodiment of the present invention, wherein the filter assembly is located between the light source and the sample plane of the microscope;

[0049] FIG4 is a schematic flow chart of a microscope imaging method provided by an exemplary embodiment of the present invention, wherein the filter assembly is located between the objective lens and the reflector of the microscope;

[0050] FIG5 is a schematic diagram of a system 100 for executing the method described in an embodiment of the present invention;

[0051] FIG6 is an imaging solution using a beam splitter to guide light to a color camera and a black and white camera respectively;

[0052] FIG7 is an imaging scheme using a movable reflective module to allow light to enter a color camera;

[0053] FIG8 is an imaging solution that uses a movable reflection module to switch the reflection mirror to allow light to enter the black and white camera.

[0054] The reference numerals include: 100 - system, 110 - microscope, 120 - computer system. DETAILED DESCRIPTION

[0055] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0056] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0057] In the scheme for forming color and black-and-white images in FIG6 , a portion of the light is directed to the color camera 6 and the remaining light is directed to the black-and-white camera 4 using a beam splitter 34. Therefore, neither the color camera 6 nor the black-and-white camera 4 can utilize 100% of the light, resulting in reduced imaging sensitivity. This is disadvantageous and unacceptable for applications of weak signal imaging such as fluorescence.

[0058] Although the color image and black and white image solutions in Figures 7 and 8 allow the color camera 6 and the black and white camera 4 to use 100% of the light to form images, they require a movable reflection module to switch between different reflectors 33, which increases the complexity and cost of the system and reduces the stability of the system imaging.

[0059] In one embodiment of the present invention, referring to FIG1 , a microscope imaging system is provided, which is configured with an illumination path and an imaging path that are interconnected, and the system includes:

[0060] a light source 1 disposed at the starting end of the illumination path and configured to provide illumination to a stage 2 of a microscope;

[0061] a filter assembly 5, configured to provide a red light filter, a green light filter, and a blue light filter in turn to the illumination path or the imaging path;

[0062] a black and white camera 4, which is provided at the end of the imaging path and is configured to generate an R-channel monochrome image, a G-channel monochrome image, and a B-channel monochrome image corresponding to the red light filter, the green light filter, and the blue light filter; and

[0063] The processor module 6 is electrically connected to the black and white camera 4 , and is configured to process the R channel monochrome image, the G channel monochrome image, and the B channel monochrome image through an image fusion algorithm to obtain a color image.

[0064] Specifically, the illumination path is defined as the light path between the light source 1 and the stage. As shown in FIG3 , the filter assembly 5 is disposed on the illumination path, specifically between the light source 1 and the sample plane of the stage 2 ;

[0065] The imaging path is defined as the optical path between the microscope's objective lens 31 and the microscope's tube lens 32. In one embodiment of the present invention, a reflector 33 is positioned along the imaging path to reduce the microscope's overall length. As shown in Figures 1 and 2 , a filter assembly 5 is positioned along the imaging path. Figure 1 shows the filter assembly 5 positioned between the objective lens and the reflector 33, while Figure 2 shows the filter assembly 5 positioned between the reflector 33 and the tube lens 32.

[0066] The illumination path and the imaging path are interconnected, so that the light emitted by the light source 1 extends from the illumination path to the imaging path. As shown in FIG1 , the light emitted by the light source 1 passes through the sample, the objective lens 31, the reflector 33, and the tube lens 32 in sequence before entering the lens of the black and white camera 4 to realize imaging of the sample.

[0067] The filter assembly 5 can be set on different optical paths as shown in Figures 1, 2, and 3. In this embodiment, the filter assembly 5 is set on the imaging path and close to the objective lens in the manner of Figure 1, that is, the distance between the filter assembly 5 and the objective lens 31 is smaller than the distance between the filter assembly 5 and the tube lens 32, which can make the beam aperture smaller. Accordingly, the filter size can be made smaller. The small filter size is conducive to the installation of the filter assembly and the filter meeting the surface accuracy requirements to improve the stability of imaging quality. The surface accuracy requirements here include the requirements for the transmission wavefront error of the filter.

[0068] The filter assembly includes a turntable and a driving mechanism, wherein the turntable is provided with at least three circumferentially arranged holes, in which a red light filter, a green light filter, and a blue light filter are respectively provided; the driving mechanism is configured to drive the turntable to rotate so that the three holes are alternately positioned on the illumination path or the imaging path, so that the red light filter, the green light filter, and the blue light filter are sequentially in the optical path: when the red light filter is in the optical path, only light within the red light wavelength range is allowed to pass through the red light filter; when the green light filter is in the optical path, only light within the green light wavelength range is allowed to pass through the green light filter; and when the blue light filter is in the optical path, only light within the blue light wavelength range is allowed to pass through the blue light filter.

[0069] In this embodiment, appropriate holes are also opened at the positions on the turntable where the filter is not installed. On the one hand, this reduces the weight of the turntable, and on the other hand, the circumferential weight balance of the turntable is taken into consideration. If necessary, the circumferential weight balance can be adjusted by local counterweights in necessary areas to reduce the jitter of the turntable when it is rotating.

[0070] Furthermore, the filter assembly provides the same filter to the illumination path or imaging path for a duration greater than or equal to the single exposure time of the black-and-white camera. When a filter reaches the optical path, it triggers the exposure of the black-and-white camera. The same filter remains in the optical path until the end of the exposure of the black-and-white camera. To achieve the above, the size / shape of the filter and the speed of the drive mechanism driving the filter need to be compatible with the exposure time of the camera.

[0071] Unlike the additional movable reflection module in Figures 7 and 8, the provision of a rotatable filter assembly in this embodiment does not affect the alignment between the original microscope light source, optical elements (objective lens 31, tube lens 32) and camera, and in this embodiment, there can be a greater error tolerance for the mechanical precision of the rotatable filter assembly. For example, the requirement for mechanical precision can be reduced by increasing the length of the filter on the rotation path.

[0072] The microscopic imaging system of this embodiment can be applied to various microscopes. A driven, rotating filter assembly can be easily added to an existing microscope system, allowing for color imaging in conjunction with a black-and-white camera. Furthermore, the imaging signal can be fully utilized in various applications, and the added filter assembly does not affect image quality compared to the original system.

[0073] In one embodiment, the red light filter, the green light filter, and the blue light filter are all strip-shaped, the widths of the red light filter, the green light filter, and the blue light filter are all greater than the optical path width of the illumination path or the imaging path, and the edges of the filters in the longitudinal direction are arc-shaped. Specifically, the arc-shaped contours of the red light filter, the green light filter, and the blue light filter are three segmented arc segments on the same circumference. Compared to circular filters, arc-shaped strip filters can improve filter utilization efficiency. That is, when the two filters have the same area, the arc-shaped strip filter can extend the time it is continuously provided in the optical path at the same drive speed. In other words, to meet the same camera exposure time, the arc-shaped strip filter can adapt to a longer camera exposure time than the circular filter, reducing the performance requirements of the camera.

[0074] In one embodiment, three holes are arranged circumferentially on the turntable, and the three holes have the same shape / size and are arranged at equal intervals. For example, the starting point where the red light filter enters the optical path is set as the initial position, and the black and white camera is triggered to expose at 0° and every 120° rotation. The single-channel monochrome images obtained by three consecutive exposures can be fused to obtain a color image.

[0075] In another embodiment, as shown in Figures 1 to 3, the turntable has four circumferentially arranged hole positions, and the fourth hole position is an empty hole position, at which no filter is placed; when the empty hole position is on the illumination path or the imaging path (the turntable does not need to rotate at this time), the black and white camera exposure can generate a black and white image.

[0076] In this embodiment, a color image can also be synthesized by obtaining monochrome images of the R / G / B channels by triggering the black and white camera to expose after a set angle. For example, if adjacent holes in the four holes are spaced 90° apart, and the starting point where the empty hole enters the optical path is used as the initial position, the black and white camera is triggered to expose after rotating 90°+n×360°, 180°+n×360°, and 270°+n×360°, respectively, where n is an integer including 0 and increases from 0.

[0077] In the above-mentioned embodiment of setting the initial position, the turntable needs to be mechanically reset before each imaging. Since the initial position in this embodiment is the position where the empty hole is located in the optical path, if a black and white image is to be generated, after completing the mechanical reset, the turntable is kept stationary and the camera exposure can generate a black and white image. If the initial position is another position, the turntable can be controlled to rotate the corresponding angle based on the angle between the initial position and the empty hole position after completing the mechanical reset so that it rotates to a position where the empty hole position is aligned with the optical path.

[0078] In an embodiment where the turntable does not require an initial position, the turntable is integrated with an encoder disk and a sensor. The encoder disk is annular and mounted outside the turntable. The processor module is configured to be electrically connected to the sensor to obtain the real-time position of the turntable. Specifically, the red light filter, the green light filter, and the blue light filter each have a different code (the annular encoder disk has different numbers of lines corresponding to the different filters, and the number of lines encodes the filter). As the turntable rotates, the encoder disk undergoes relative displacement with the sensor, causing the encoder disk to enter the sensor's detection area slightly before the corresponding filter enters the optical path. By detecting the coded signal from the sensor and the preset association between the codes and the filters, the processor can determine whether the filter currently entering the optical path is a red light filter, a green light filter, or a blue light filter. In a specific embodiment, for example, the code corresponding to the red light filter is represented by one grid, the code corresponding to the green light filter is represented by two grids, and the code corresponding to the blue light filter is represented by three grids. Each grid allows the sensor to detect a signal once. If the sensor detects two more signals within a preset time period (less than the time it takes to switch to the next code) after detecting the first signal, the current position of the turntable is identified as the location of the blue light filter. If the sensor detects another signal within a preset time period (less than the time it takes to switch to the next code) after detecting the first signal, the current position of the turntable is identified as the location of the green light filter. If the sensor does not detect any signal within a preset time period (less than the time it takes to switch to the next code) after detecting the first signal, the current position of the turntable is identified as the location of the red light filter. This eliminates the need to set a starting position, eliminates the need to restrict the positional relationship between the three or more holes on the turntable, and eliminates the need to restrict the clockwise or counterclockwise rotation direction of the turntable.

[0079] In the above embodiment where the turntable does not need to be set to an initial position, if a black and white image is required, two methods can be adopted: Method 1: Different codes are set on the positions corresponding to the empty holes on the encoder disk. For example, the corresponding codes are represented by four grids, and the empty holes are identified by identifying the red light filter / green light filter / blue light filter. That is, each time the turntable is turned to an empty hole position and aligned with the light path, the black and white camera is triggered to expose;

[0080] Method 2: First, control the turntable to slow down to a lower speed, and then when the code of the empty hole position is recognized next time, use electrical braking or mechanical braking to stop the turntable at the position where the empty hole position is aligned with the optical path.

[0081] The red light filter has a central wavelength between 600 and 700 nm; the green light filter has a central wavelength between 500 and 600 nm; and the blue light filter has a central wavelength between 400 and 500 nm. The light source is a white light source, and the wavelength of the light it emits covers a range of 400 to 700 nm. In this embodiment, the wavelengths and transmittances of the red, green, and blue light filters are set based on the spectrum of the light source, the quantum efficiency of the black-and-white camera, and optionally in combination with the spectral transmittance of the optical components in the microscope, so that the color image achieves a predetermined color reproduction degree. In a specific embodiment, the red light filter has a central wavelength of 660 nm, a half-bandwidth between 18 and 22 nm, and a transmittance greater than or equal to 90%; the green light filter has a central wavelength of 540 nm, a half-bandwidth between 18 and 22 nm, and a transmittance between 40% and 60%; and the blue light filter has a central wavelength of 460 nm, a half-bandwidth between 18 and 22 nm, and a transmittance between 40% and 60%.

[0082] In order to generate a video, when the camera generates an R channel monochrome image, a G channel monochrome image, and a B channel monochrome image, each monochrome image is assigned a serial number in sequence according to the order of imaging. The processor module reads the serial number of the image, and sequentially fuses the three monochrome images with adjacent serial numbers to obtain a frame of color image, and generates a color video with multiple frames of color images. Specifically, the monochrome images of each channel are sorted according to the serial number, so that one of the three adjacent channel monochrome images must be an R channel monochrome image, another is a G channel monochrome image, and the third is a B channel monochrome image. Taking the order of RGB as an example, the sorted images are R1, G1, B1, R2, G2, and B2, sorted in sequence. The color video can be a real-time color video, and its color video frame rate matches the frame rate of the R channel monochrome image, the G channel monochrome image, and the B channel monochrome image. There are three ways to calculate the frame rate of a video:

[0083] Method 1: Combine R1, G1, and B1 into one frame of video image, and combine R2, G2, and B2 into one frame of video image;

[0084] Method 2: R1, G1, and B1 are combined into one video frame, G1, B1, and R2 are combined into a second video frame, B1, R2, and G2 are combined into a third video frame, and R2, G2, and B2 are combined into a fourth video frame.

[0085] Method three: combine R1, G1, and B1 into one frame of video image, combine B1, R2, and G2 (or G1, B1, and R2) into a second frame of video image, and combine R2, G2, and B2 into a third frame of video image.

[0086] The frame rate of the second method is greater than that of the third method, and the frame rate of the third method is greater than that of the first method.

[0087] Obviously, it should be understood that in addition to generating real-time video, the system in this embodiment can also use a storage device to save each R channel monochrome image, G channel monochrome image, and B channel monochrome image with assigned serial numbers to generate non-real-time video. For non-real-time video, its video frame rate can be further improved.

[0088] In one embodiment of the present invention, a microscope imaging method is provided, as shown in FIG4 , the imaging method comprising the following steps:

[0089] Turn on the light source so that it illuminates the microscope stage;

[0090] Driving the filter assembly to rotate so that the red light filter, the green light filter and the blue light filter of the filter assembly arrive at the light path in turn;

[0091] Controlling a black and white camera to expose the red light filter, the green light filter, and the blue light filter when they arrive in the optical path, respectively, to obtain an R channel monochrome image, a G channel monochrome image, and a B channel monochrome image;

[0092] The R channel monochrome image, the G channel monochrome image, and the B channel monochrome image are processed by an image fusion algorithm to obtain a color image.

[0093] Furthermore, serial numbers are assigned to the R channel monochrome image, the G channel monochrome image, and the B channel monochrome image, and video frames of the color video are generated according to the serial numbers.

[0094] Furthermore, the duration of each of the red light filter, the green light filter or the blue light filter on the optical path is greater than or equal to a single exposure time of the black and white camera.

[0095] Furthermore, the black and white camera is controlled to perform exposure without a red light filter, a green light filter and a blue light filter in the optical path to generate a black and white image.

[0096] The microscope imaging method provided in the embodiment of the present invention and the microscope imaging system provided in the above embodiment belong to the same inventive concept, and the contents of the microscope imaging system embodiment are incorporated into the present imaging method embodiment by reference in full.

[0097] According to another aspect of the present invention, an electronic device is provided, comprising a processor and a memory, wherein the memory is used to store program instructions, and the processor is configured to run the program instructions, characterized in that the program instructions are run to perform the steps of the method described above.

[0098] According to another aspect of the present invention, a computer-readable storage medium is provided for storing program instructions, wherein the program instructions are configured to be called to execute the steps of the above-mentioned method.

[0099] According to another aspect of the present invention, a computer program product is provided, comprising a readable and stored computer program, wherein the computer program comprises program instructions, and is characterized in that when the program instructions are executed on a computer device, the computer device executes the steps of the above-mentioned method.

[0100] Some embodiments relate to a microscope including a device for imaging microscopic images. Optionally, the microscope can be part of or connected to a system that performs the method flow shown in Figure 4. Figure 5 shows a schematic diagram of a system 100 configured to perform the methods described herein. System 100 includes a microscope 110 and a computer system 120. Microscope 110 is configured to capture images and is connected to computer system 120. Computer system 120 is configured to perform at least a portion of the methods described herein. Computer system 120 can be configured to execute a machine learning algorithm. Computer system 120 and microscope 110 can be separate entities, but can also be integrated into a common housing. Computer system 120 can be part of the central processing system of microscope 110 and / or, computer system 120 can be part of a subcomponent of microscope 110, such as a sensor, actuator, camera, or lighting unit of microscope 110.

[0101] Computer system 120 can be a local computer device (e.g., a personal computer, laptop, tablet, or mobile phone) having one or more processors and one or more storage devices, or it can be a distributed computer system (e.g., having one or more processors and one or more storage devices distributed across various locations, such as a local client and / or one or more remote server locations and / or data centers). Computer system 120 can include any circuitry or combination of circuits. In one embodiment, computer system 120 can include one or more processors of any type. As used herein, a processor can refer to any type of computing circuitry, such as, but not limited to, a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), a multi-core processor, a field programmable gate array (FPGA) such as a microscope or microscope component (e.g., a camera), or any other type of processor or processing circuitry. Other types of circuitry that can be included in computer system 120 can include custom circuitry, application-specific integrated circuits (ASICs), and the like, such as one or more circuits (e.g., communication circuitry) used in wireless devices such as mobile phones, tablets, laptops, two-way radios, and similar electronic systems. The computer system 120 may include one or more storage devices, which may include one or more storage elements suitable for a particular application, such as main memory in the form of random access memory (RAM), one or more hard disk drives, and / or one or more drives for handling removable media such as compact disks (CDs), flash memory cards, digital video disks (DVDs), etc. The computer system 120 may also include a display device, one or more speakers, and a keyboard and / or controller, which may include a mouse, a trackball, a touch screen, a voice recognition device, or any other device that allows a system user to input information to or receive information from the computer system 120.

[0102] In one embodiment of the present invention, an electronic device is provided, comprising a processor and a memory, wherein the memory is used to store program instructions, and the processor is configured to run the program instructions, and the program instructions are run to perform the steps performed in the above method embodiment.

[0103] In one embodiment of the present invention, a computer-readable storage medium is provided for storing program instructions, wherein the program instructions are configured to be called to execute the steps performed in the above method embodiment.

[0104] In one embodiment of the present invention, a computer program product is provided, including a readable and stored computer program, wherein the computer program includes program instructions. When the program instructions are executed on a computer device, the computer device executes the steps executed in the above method embodiment.

[0105] It should be noted that the above-mentioned image processing method, image stitching device, microscopic imaging system, electronic device, computer-readable storage medium, computer program product embodiments and the image stitching method embodiments belong to the same inventive concept, and the entire content of the image stitching method embodiments are incorporated into the image processing method, image stitching device, microscopic imaging system, electronic device, computer-readable storage medium, and computer program product embodiments by reference.

[0106] Some or all of the method steps may be performed by (or using) a hardware device (eg, a processor, a microprocessor, a programmable computer, or an electronic circuit). In some embodiments, such a device may perform one or more of the most important method steps.

[0107] Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. This implementation may be performed using a non-transitory storage medium (such as a digital storage medium, e.g., a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or FLASH) having stored thereon electronically readable control signals that cooperate (or are capable of cooperating) with a programmable computer system to execute the corresponding method. Thus, the digital storage medium may be computer-readable.

[0108] Some embodiments of the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0109] Generally, embodiments of the present invention can be implemented as a computer program product having a program code, which is operable to perform one of the methods when the computer program product runs on a computer. The program code can, for example, be stored on a machine-readable carrier.

[0110] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0111] In other words, an exemplary embodiment of the present invention is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0112] Therefore, a further embodiment of the present invention is a storage medium (or data carrier or computer-readable medium) comprising a computer program stored thereon, which, when executed by a processor, is configured to perform one of the methods described herein. The data carrier, digital storage medium, or recorded medium is typically tangible and / or non-transitory. A further embodiment of the present invention is an apparatus as described herein, comprising a processor and a storage medium.

[0113] Therefore, a further embodiment of the present invention is a data stream or a signal sequence representing the computer program for performing one of the methods described herein. The data stream or signal sequence can be configured to be transmitted via a data communication connection, such as the Internet.

[0114] A further embodiment comprises a processing means, for example a computer or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0115] A further embodiment comprises a computer on which is installed the computer program for performing one of the methods described herein.

[0116] Yet another embodiment of the present invention includes an apparatus or system configured to transmit (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, or the like. The apparatus or system may, for example, include a file server for transmitting the computer program to the receiver.

[0117] In some embodiments, a programmable logic device (e.g., a field programmable gate array) can be used to perform some or all of the functions of the methods described herein. In some embodiments, the field programmable gate array can cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware device.

[0118] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0119] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ".

[0120] The above description is only a specific embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of this application. These improvements and modifications should also be regarded as the scope of protection of this application.

Claims

1. A microscope imaging system, characterized in that, The system includes: a light source configured to provide illumination to the stage of a microscope; a filter assembly configured to alternately provide a red filter, a green filter, and a blue filter to the illumination path or the imaging path of the microscope; a black-and-white camera disposed at the end of the imaging path and configured to generate an R-channel monochromatic image, a G-channel monochromatic image, and a B-channel monochromatic image corresponding to the red filter, the green filter, and the blue filter; and a processor module electrically connected to the black-and-white camera and configured to process the R-channel monochromatic image, the G-channel monochromatic image, and the B-channel monochromatic image through an image fusion algorithm to obtain a color image.

2. The microscope imaging system according to claim 1, wherein The duration for which the filter assembly provides the same filter to the illumination path or the imaging path is greater than or equal to the single exposure time of the black-and-white camera.

3. The microscope imaging system according to claim 1 or 2, characterized in that, The filter assembly includes a turntable and a driving mechanism. The turntable is configured with at least three apertures arranged circumferentially, and a red filter, a green filter, and a blue filter are respectively disposed in the apertures. The driving mechanism is configured to drive the turntable to rotate so that the three apertures alternately reach the illumination path or the imaging path.

4. The microscope imaging system according to claim 3, wherein, The turntable is integrally provided with an encoding disk and a sensor. The processor module is configured to be electrically connected to the sensor to obtain the real-time position of the turntable.

5. The microscope imaging system according to claim 3, wherein The turntable is further provided with a fourth aperture, and the fourth aperture is an empty aperture; The black-and-white camera is further configured to generate a black-and-white image when the empty aperture reaches the illumination path or the imaging path.

6. The microscope imaging system according to any one of claims 1 to 5, characterized in that It further includes an objective lens and a tube lens, which are sequentially arranged on the optical path between the light source and the black-and-white camera; The filter assembly is disposed between the objective lens and the tube lens, and the distance between the filter assembly and the objective lens is less than the distance between the filter assembly and the tube lens.

7. The microscope imaging system according to any one of claims 1 to 6, characterized in that, The central wavelength of the red filter is between 600 and 700 nm; and / or, the central wavelength of the green filter is between 500 and 600 nm; and / or, the central wavelength of the blue filter is between 400 and 500 nm.

8. The microscope imaging system according to any one of claims 1 to 7, characterized in that The red filter, the green filter, and the blue filter are all strip-shaped members.

9. The microscope imaging system according to any one of claims 1 to 8, characterized in that, The widths of the red filter, the green filter, and the blue filter are greater than the optical path width of the illumination path or the imaging path, and the edges in the length direction of the filter are arc-shaped.

10. The microscope imaging system according to any one of claims 1 to 9, characterized in that, Each R-channel monochromatic image, G-channel monochromatic image, and B-channel monochromatic image is sequentially assigned a serial number according to the imaging sequence; The processor module reads the serial numbers of the images, and sequentially fuses three adjacent monochromatic images to obtain a frame of color image, and generates a color video with multiple frames of color images.

11. The microscope imaging system according to claim 10, characterized in that, The color video is a real-time color video, and the frame rate of the color video matches the frame rates of the R-channel monochromatic image, the G-channel monochromatic image, and the B-channel monochromatic image.

12. The microscope imaging system according to any one of claims 1 to 11, characterized in that, The light source is a white light source.

13. A microscope imaging method, characterized in that, It includes the following steps: Turn on the light source to provide illumination to the stage of the microscope; Drive the filter assembly to rotate so that the red filter, the green filter, and the blue filter of the filter assembly alternately reach the optical path of the microscope; Control a black-and-white camera to perform exposure when the red light filter, green light filter, and blue light filter reach the optical path respectively, and obtain an R-channel monochrome image, a G-channel monochrome image, and a B-channel monochrome image respectively; Process the R-channel monochrome image, G-channel monochrome image, and B-channel monochrome image through an image fusion algorithm to obtain a color image.

14. The microscope imaging method according to claim 13, characterized in that, Assign serial numbers to the R-channel monochrome image, G-channel monochrome image, and B-channel monochrome image, and generate video frames of a color video according to the serial numbers.

15. The microscope imaging method according to claim 13 or 14, characterized in that, The duration of the red light filter, green light filter, or blue light filter on the optical path is greater than or equal to the single exposure time of the black-and-white camera.

16. The microscope imaging method according to any one of claims 13 to 15, characterized in that, Control the black-and-white camera to perform exposure when a blank hole position of the filter assembly reaches the optical path to generate a black-and-white image.

17. The microscope imaging method according to any one of claims 13 to 16, characterized in that, Based on the spectrum of the light source and the quantum efficiency of the black-and-white camera, set the wavelengths and transmittances of the red light filter, green light filter, and blue light filter so that the color image reaches a preset color restoration degree; Alternatively, based on the spectrum of the light source, the quantum efficiency of the black-and-white camera, and the spectral transmittance of the optical elements in the microscope, set the wavelengths and transmittances of the red light filter, green light filter, and blue light filter so that the color image reaches a preset color restoration degree.

18. An electronic device includes a processor and a memory, wherein, The memory is used to store program instructions, and the processor is configured to run the program instructions. It is characterized in that the program instructions are run to execute the steps of the method according to any one of claims 13 to 17.

19. A computer-readable storage medium for storing program instructions, characterized in that, The program instructions are configured to be called to execute the steps of the method according to any one of claims 13 to 17.

20. A computer program product, comprising a computer program stored in a readable storage medium, the computer program comprising program instructions, characterized in that, When the program instructions run on a computer device, the computer device executes the steps of the method according to any one of claims 13 to 17.

Images