Image processing system and image processing method

Abstract

To quickly perform the adjustment of a system required according to observation conditions.SOLUTION: An image processing system 1 is provided with a scanning unit 105, a PPD 111, a memory 12 storing a plurality of tables LUTs 13-1, 13-2 and 13-3 in which intensity gradation values and output gradation values are associated, and a processor 11. The processor 11 generates first image data in which the intensity gradation value indicated by a light intensity signal is a pixel value of constituent pixels based on the light intensity signal output from the PPD 111 and a scanning position by the scanning unit 105, converts the first image data based on one selected from the plurality of tables stored in the memory 12, generates second image data in which the output gradation value is the pixel value of the constituent pixels, and displays an image represented by the second image data on a display device 20.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The disclosure of this specification relates to an image processing system, and more particularly to an image processing system including a Pixelated Photon Detector (hereinafter, referred to as a PPD). [Background technology]

[0002] A technique is known that allows easy adjustment of image contrast in an image processing system equipped with a PPD, such as a laser scanning microscope system. For example, this technique adjusts the contrast of an image generated based on a light intensity signal detected by a PPD and a scanning position of a scanner using a lookup table (LUT), and displays the adjusted image. There is also a known technique that updates the LUT according to a setting of a range of gradations to be identified (identification range) in an image before adjustment, so that the identification range falls within the entire display gradation range of a display device that displays the adjusted image (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7291808 Summary of the Invention [Problem to be solved by the invention]

[0004] Recently, the detection element used to detect the intensity of light is changing from PMT (Photomultiplier Tube) to PPD and others. When PPD is used as the detection element in a laser scanning microscope, the applied high voltage HV (High Voltage) required for PMT is no longer necessary. However, because of this, the method of adjusting the brightness of the acquired image by changing the output level from the detection element by adjusting the sensitivity of the detection element can no longer be used, and the only method available is to adjust the irradiation intensity (laser power) of the laser light irradiated on the sample.

[0005] Incidentally, when imaging a sample using a laser scanning microscope, image search (sample search) work such as adjusting imaging conditions and focusing is performed in advance. Since the irradiation of laser light by a laser scanning microscope can cause discoloration of the sample, it is desirable to make the laser power as weak as possible during the above-mentioned work compared to normal imaging, and it is also desirable to keep the work time short.

[0006] On the other hand, when taking a formal image of a sample, the irradiation intensity of the laser light may be increased even more than in normal imaging, even though it may cause the sample to fade, in order to obtain a good image with high contrast and many gradations. It is clear that it is desirable to complete imaging under such irradiation intensity in a short time.

[0007] Thus, when acquiring an image using a laser scanning microscope, the irradiation intensity of the laser light is changed depending on the purpose of observing the acquired image. When the irradiation intensity is changed, the intensity of the light that the light receiving element of the laser scanning microscope receives from the sample changes. With the technology described above, it is necessary to change the setting of the recognition range and update the LUT every time the irradiation intensity is changed, and such setting changes are time-consuming.

[0008] In view of the above circumstances, an object of one aspect of the present invention is to enable quick adjustment of a system that is required in accordance with observation conditions. [Means for solving the problem]

[0009] An image processing system according to one embodiment of the present invention includes a scanner, a PPD, a memory in which a plurality of tables that correspond to intensity gradation values ​​and output gradation values ​​are stored, and a processor configured to perform the following processing: generate first image data in which the intensity gradation value indicated by the light intensity signal is set as the pixel value of a constituent pixel based on a light intensity signal output from the PPD and a scanning position by the scanner, convert the first image data based on one of the tables selected from the plurality of tables stored in the memory to generate second image data in which the output gradation value is set as the pixel value of a constituent pixel, and display an image represented by the second image data on a display unit.

[0010] An image processing method according to one embodiment of the present invention generates first image data based on a light intensity signal output from a PPD and a scanning position by a scanner, in which the intensity gradation value indicated by the light intensity signal is used as the pixel value of a constituent pixel, converts the first image data based on one selected from a plurality of tables stored in a memory, which tables correspond to the intensity gradation values ​​and output gradation values, to generate second image data in which the output gradation value is used as the pixel value of a constituent pixel, and displays an image represented by the second image data on a display unit. Effect of the Invention

[0011] According to the above aspect, it becomes possible to quickly adjust the system according to the observation conditions. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 illustrates an example of the configuration of an image processing system according to an embodiment. [Diagram 2] 11 is a graph showing an example of the relationship between intensity gradation values ​​and output gradation values ​​of each LUT. [Diagram 3] 11 is a flowchart showing the processing contents of processing performed by a processor. [Figure 4A] 1 is a first example (part 1) of a display screen on a display device. [Figure 4B] This is the first example (part 2) of a display screen on a display device. [Figure 4C] This is a first example (part 3) of a display screen on a display device. [Diagram 5] 13 is a second example of a display screen on the display device. [Figure 6A] 13 is a third example (part 1) of a display screen on a display device. [Figure 6B] This is the third example (part 2) of the display screen on the display device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, the embodiments will be described in detail with reference to the drawings.

[0014] FIG. 1 is a diagram showing an example of the configuration of an image processing system 1 according to an embodiment.

[0015] 1, the image processing system 1 includes an optical instrument 100, a computer 10 connected to the optical instrument 100, and a display device 20 and an input device 30 connected to the computer 10. The optical instrument 100 is, for example, the main body of a laser scanning microscope, and the image processing system 1 is, for example, a laser scanning microscope system.

[0016] The optical device 100 includes a scanner that scans a sample with light and a PPD 111. The computer 10 adjusts the contrast of an image generated based on a light intensity signal detected by the PPD 111 and the scanning position of the scanner, and displays the adjusted image on the display device 20. The computer 10 may also control the optical device 100 based on information input by a user using the input device 30.

[0017] In the optical device 100, the scan unit 105 scans the sample on the stage 101 with laser light emitted from the laser 107, and the PPD 111 detects the light from the sample that has entered via a confocal optical system including a confocal aperture 109. Then, the optical device 100 outputs to the computer 10 an intensity signal of the light from the sample detected by the PPD 111 and the scanning position by the scan unit 105.

[0018] The optical device 100 includes a stage 101, an objective lens 102, a focusing device 103, a relay optical system 104, a scan unit 105, a beam splitter 106, a laser 107 as a light source, an imaging lens 108, a confocal aperture 109, and a lens 110. The optical device 100 further includes a PPD 111, an analog amplifier 112, an AD converter 113, and a digital amplifier 114.

[0019] The laser light emitted from the laser 107 and reflected by the beam splitter 106 passes through the scan unit 105 and the relay optical system 104 and enters the objective lens 102. The beam splitter 106 is, for example, a dichroic mirror that reflects the laser light. The objective lens 102 focuses the laser light on a sample placed on the stage 101 and forms a light spot on the sample.

[0020] In the sample irradiated with the laser light, fluorescent substances are excited and emit fluorescence with a wavelength different from that of the laser light. This fluorescence passes through the objective lens 102, relay optical system 104, and beam splitter 106 on which it is incident via the scan unit 105, and is focused on the confocal aperture 109 by the imaging lens 108. A confocal pinhole is formed in the confocal aperture 109 at a position optically conjugate with the focal position on the front side (sample side) of the objective lens 102. Therefore, the fluorescence generated at a position other than the position where the light spot is formed is blocked by the confocal aperture 109, and only the fluorescence generated at the position where the light spot is formed passes through the confocal aperture 109 and is detected by the PPD 111 via the lens 110.

[0021] The PPD 111 detects light from the sample and outputs a light intensity signal according to the number of incident photons, i.e., an intensity signal indicating the intensity of the light. The light intensity signal output from the PPD 111 is amplified by an analog amplifier 112 and a digital amplifier 114 before and after sampling by an AD converter 113. The optical device 100 outputs the intensity signal amplified by the analog amplifier 112 and the digital amplifier 114 and the scanning position corresponding to the intensity signal to the computer 10.

[0022] In this embodiment, the intensity signal output from the optical device 100 to the computer 10 is 16 bits long. That is, the number of gradations of the intensity gradation value indicated by this intensity signal is 65536. Of course, the bit length of the intensity gradation value may be other than 16 bits.

[0023] The scan unit 105 includes at least one pair of scanners that scan the sample in mutually orthogonal directions. The scan unit 105 may include, for example, two galvanometer scanners, or may include a galvanometer scanner and a resonant scanner. The scanner included in the scan unit 105 is not limited to a galvanometer scanner or a resonant scanner. Any configuration may be used as long as the sample is scanned by deflecting light, and other light deflectors such as an acousto-optical deflector (AOD) may be used for the scanner. By changing the swing angle of the scanner and thereby changing the direction in which the laser light is deflected, it is possible to change the angle of the laser light with respect to the optical axis on the pupil plane of the objective lens 102. This allows the position of the light spot to be moved in a direction orthogonal to the optical axis of the objective lens 102. The optical device 100 can scan the sample two-dimensionally with the laser light by controlling the scan unit 105 in response to an instruction from the computer 10, and can obtain information required for constructing a two-dimensional image of the sample.

[0024] The computer 10 includes at least one processor 11 and at least one memory 12 .

[0025] The processor 11 is configured to perform a generation process for generating image data based on an intensity signal output from the optical device 100 and a scanning position corresponding to the intensity signal. More specifically, this generation process is a process for generating image data representing an image in which an intensity gradation value representing the intensity of light from a sample indicated by the intensity signal is set as a pixel value of a constituent pixel corresponding to the scanning position.

[0026] In the following description, the image data generated by this generation process will be referred to as "first image data."

[0027] The processor 11 is further configured to perform a selection instruction process to acquire an instruction to select one of the LUTs 13-1, 13-2, and 13-3 used for adjusting an image. The LUTs 13-1, 13-2, and 13-3 are tables that respectively associate intensity gradation values ​​representing the intensity of light with output gradation values, and are stored in the memory 12 in advance.

[0028] The processor 11 is further configured to perform a conversion process for converting the first image data using one selected from among the LUTs 13-1, 13-2, and 13-3 according to the instruction, to generate image data in which the output gradation value is the pixel value of the constituent pixel. More specifically, this conversion process converts the light intensity gradation value, which is the pixel value of each constituent pixel in the first image data, into the output gradation value associated with the one selected from among the LUTs 13-1, 13-2, and 13-3.

[0029] In the following description, the image data generated from the first image data by this conversion process will be referred to as "second image data."

[0030] The processor 11 is further configured to perform a display process for displaying, on the display device 20, an image represented by the generated second image data.

[0031] The processor 11 may perform the above-mentioned processes by executing software stored in the memory 12, or may perform the above-mentioned processes by hardware processing. Alternatively, the processor 11 may perform the above-mentioned processes by a combination of software processing and hardware processing. The software executed by the processor 11 is stored in a computer-readable medium. The processor 11 includes any electric circuit, and may include at least one of a circuit for processing digital signals and a circuit for processing analog signals. The processor 11 may include, for example, one or more integrated circuits (ICs) on a circuit board, and may further include one or more electronic components. The integrated circuit may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or the like. The integrated circuit may be, for example, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or the like.

[0032] The memory 12 includes a computer-readable medium storing software executed by the processor 11. In this specification, the computer-readable medium is a non-transitory computer-readable medium. The memory 12 may include, for example, one or more arbitrary semiconductor memories, and one or more other storage devices. The semiconductor memory may include, for example, a volatile memory such as a random access memory (RAM), and a non-volatile memory such as a read only memory (ROM), a programmable ROM, and a flash memory. The RAM may include, for example, a dynamic random access memory (DRAM), a static random access memory (SRAM), and the like. The other storage devices may include, for example, a magnetic storage device including, for example, a magnetic disk as a computer-readable medium, and an optical storage device including, for example, an optical disk as a computer-readable medium.

[0033] The display device 20 is an example of a display unit of the image processing system 1. The display device 20 is, for example, a liquid crystal display, an organic EL display, a CRT (Cathode Ray Tube) display, etc. On the display device 20, for example, a setting screen is displayed.

[0034] The input device 30 is an example of an input unit of the image processing system 1. The input device 30 outputs an operation signal corresponding to an operation by a user to the computer 10. The input device 30 is, for example, a keyboard, but may also include a mouse, a joystick, a touch panel, a stylus, or the like.

[0035] The image processing system 1 shown in FIG. 1 includes the above-mentioned components.

[0036] Next, the LUTs 13-1, 13-2, and 13-3 will be described in more detail.

[0037] In certain applications, such as observations that combine dyes with different intensities, or observations where there is a large difference in brightness depending on the observation position, such as depth (Z-direction position) or XY-direction position, it may be desirable to have as many as 65,536 gradations (16-bit length) in the acquired image. For example, when various applications including those mentioned above are taken into consideration comprehensively, the number of gradations required in the acquired image, converted into the number of photons incident on the PPD111, is 2000 (≒ 2048 (2 11 The PPD111 has a dynamic range sufficient to obtain an image with such an extremely high number of gray scales.

[0038] On the other hand, during the image search process described above, even if the image is saturated, it is sufficient to be able to visually confirm the presence or absence of the image of the sample from the image, so there is no problem with the desired number of gradations for the acquired image being as low as 256 gradations (equivalent to 8 bit length and 8 photons). However, during this process, if the intensity of the laser light irradiated onto the sample is weakened to suppress fading of the sample, the acquired image will be dark.

[0039] The smaller range of the 65536 intensity gradation values ​​(range of gradation values ​​0 to 255) may be used as the range in which the output gradation value changes linearly with respect to the intensity gradation value. In this way, even if the intensity gradation value is distributed in a low region due to weak light from the sample, the image of the sample can be easily viewed while suppressing fading when searching for the image.

[0040] Furthermore, in general applications, excluding the specific cases mentioned above, even if image analysis processing may be performed later, the desired number of gradations for the acquired image is 4096 gradations (12-bit length, equivalent to 128 photons) is sufficient.

[0041] Thus, it is necessary to adjust the number of gradations of the acquired image in image search work, general applications, etc. For example, if the acquired image has 65,536 gradations, an image with 1 / 256 gradations is sufficient for the above-mentioned image search work, and an image with 1 / 16 gradations is sufficient for the above-mentioned general applications.

[0042] The LUTs 13-1, 13-2, and 13-3 in this embodiment are used to adjust the number of gradations of an image obtained when the intensity of light from a sample is weak.

[0043] Here, a description will be given of Fig. 2. Fig. 2 is a graph showing an example of the relationship between the intensity gradation value and the output gradation value for each of the LUTs 13-1, 13-2, and 13-3.

[0044] In Fig. 2, graphs G1, G2, and G3 respectively show the relationship between the intensity gradation value indicating the light intensity and the output gradation value associated with the intensity gradation value for LUTs 13-1, 13-2, and 13-3. Note that in Fig. 2, the horizontal axis shows the intensity gradation value (the gradation value indicating the intensity indicated by the intensity signal output from the optical device 100), and the vertical axis shows the output gradation value.

[0045] The LUT 13-1 having the relationship of graph G1 is a table used to obtain an image with the fewest number of gradations.

[0046] In graph G1, the output gradation value when the intensity gradation value is a predetermined upper limit value I1 is V, and in the range of the intensity gradation value below I1, the intensity gradation value and the output gradation value are in a proportional relationship. Also, in this graph G1, all intensity gradation values ​​in the range exceeding I1 correspond to the output gradation value V when the intensity gradation value is I1. Due to this relationship, for example, when the value of I1 is 255, an intensity gradation value having a gradation number of 16 bits (65536 gradations) is converted into an output gradation value of 256 gradations (8 bits).

[0047] By performing image conversion using such LUT13-1, it becomes possible to display an image of the sample with a certain degree of contrast without exceeding the upper limit of the gradation range that can be displayed by the display device 20, even if the intensity of the laser light irradiated onto the sample is weakened.

[0048] The LUT 13-2 having the relationship of graph G2 is a table used to obtain an image with a greater number of gradations than the LUT 13-1.

[0049] In the graph G2, the output gradation value when the intensity gradation value is a predetermined upper limit value I2 (>I1) is V, and in the range of the intensity gradation value below I2, the intensity gradation value and the output gradation value are in a proportional relationship. In addition, this graph G2 has a relationship in which all intensity gradation values ​​in the range exceeding I2 correspond to the output gradation value V when the intensity gradation value is I2. Due to this relationship, for example, when the value of I2 is 4095, the intensity gradation value having a gradation number of 16 bits (65536 gradations) is converted into an output gradation value of 4096 gradations (12 bits). The gradation number of this output gradation value is, for example, a gradation number that does not impair the gradation that can express each color of the three primary colors of light (RGB) in 8 bits in a standard display device. The gradation number of 4096 is the same as the gradation number of a laser scanning microscope system using a PMT, and is easy to be familiar with users of this system. Furthermore, this number of gradations corresponds to 128 photons when converted into the number of photons incident on the PPD 111, which is a realistic value for a laser scanning microscope system using a PPD.

[0050] The LUT 13-3 having the relationship of graph G3 is a table used to obtain an image with an even greater number of gradations than the LUT 13-2.

[0051] In the graph G3, the output gradation value when the intensity gradation value is I3 (>I2) is V, and in the range of the intensity gradation value below I3, the intensity gradation value and the output gradation value are in a proportional relationship. In addition, this graph G3 has a relationship in which all intensity gradation values ​​in the range exceeding I3 correspond to the output gradation value V when the intensity gradation value is I3. Due to this relationship, for example, when the value of I3 is 16383, the intensity gradation value having a gradation number of 16 bits (65536 gradations) is converted into an output gradation value of 16384 gradations (14 bits). Note that the gradation number of 16384 is a gradation number at which gradation loss can be ignored when post-processing image data such as a super-resolution image or an HDR (High Dynamic Range) image.

[0052] In addition, in graph G3, for example, the value of I3 may be set to 65535, and the intensity gradation value having 16-bit gradation may be directly converted into an output gradation value of 65536 gradations (16 bits).

[0053] In addition, the relationship between the intensity gradation value and the output gradation value for the LUTs 13-1, 13-2, and 13-3 is not limited to the relationship shown in the graphs G1, G2, and G3 in FIG. 2. For example, the output gradation value corresponding to the upper limit I1 of the intensity gradation value in the graph G1, the output gradation value corresponding to the upper limit I2 of the intensity gradation value in the graph G2, and the output gradation value corresponding to the upper limit I3 of the intensity gradation value in the graph G3 may be different values. In addition, for example, in a range in which the intensity gradation value in the graphs G1, G2, and G3 is equal to or less than a predetermined upper limit value, the output gradation value may monotonically increase with an increase in the intensity gradation value. In addition, for example, the output gradation value when the intensity gradation value is zero may be associated with a value greater than zero.

[0054] Also, for example, the output gradation value associated with an intensity gradation value close to 0 (for example, gradation value 10) may be set to 0. In this case, the upper limit values ​​I1, I2, and I3 of the graphs G1, G2, and G3 in Fig. 2 are changed and increased (for example, increased by 10 gradations), and the output gradation value is set to monotonically increase with respect to the intensity gradation value between the lower limit value and the upper limit value. In this way, the number of gradations (number of bits) of the output gradation value can be maintained, and when an offset component is included in the intensity signal output from the PPD 111, the offset component can be effectively reduced.

[0055] In the following description, the conversion process performed by the processor 11 using the LUT 13-1 having the relationship of graph G1 will be referred to as the "conversion process with luminance range A." Similarly, the conversion process performed by the processor 11 using the LUT 13-2 having the relationship of graph G2 will be referred to as the "conversion process with luminance range B." Moreover, the conversion process performed by the processor 11 using the LUT 13-3 having the relationship of graph G3 will be referred to as the "conversion process with luminance range C."

[0056] Next, a specific example of processing using the above-described image processing system 1 will be described.

[0057] Fig. 3 is a flowchart showing the process contents performed by the processor 11 included in the computer 10. Fig. 4A, Fig. 4B, and Fig. 4C each show a first example of a display screen on the display device 20.

[0058] When the image processing system 1 is started to start the processing of FIG. 3, first, in S101, processing for displaying the icon button group 40 on the screen of the display device 20 is performed.

[0059] The computer 10 employs a GUI (Graphical User Interface) using a window system, and in the process of S101, an icon button group 40 is displayed in a window for the image processing system 1.

[0060] The icon button group 40 includes a brightness range selection button 41. The brightness range selection button 41 has three icon buttons, each of which corresponds to a brightness range A, B, and C, respectively.

[0061] Next, in S102, a process is performed to select the brightness range A of the three brightness ranges as the default brightness range. Note that in this process, the brightness range B or the brightness range C of the three brightness ranges may be selected as the default brightness range.

[0062] Fig. 4A is a display example when an icon button corresponding to brightness range A of the brightness range selection buttons 41 is selected. Fig. 4B and Fig. 4C are display examples when icon buttons corresponding to brightness ranges B and C of the brightness range selection buttons 41 are selected, respectively. In this way, one of the three icon buttons is selected as the brightness range selection button 41.

[0063] In S103, a process is performed in which the brightness range display section 50 is displayed on the screen of the display device 20 with display contents according to the brightness range selection status at this time of the process.

[0064] In S104, the above-mentioned generation process is performed, and the first image data is generated based on the intensity signal received from the optical device 100 and the scanning position corresponding to the intensity signal. The generated first image data may be stored in the memory 12 as necessary.

[0065] In S105, the conversion process described above is performed, and the second image data is generated from the first image data generated in the most recent process of S104 using one of LUTs 13-1, 13-2, and 13-3 that corresponds to the luminance range selected at the time of this process. For example, when the process of S105 is performed immediately after the process of S102 described above, the conversion process is performed using LUT 13-1 that corresponds to the luminance range A.

[0066] In the display example of Fig. 4A, the brightness range display section 50 is displayed within the window when the brightness range A is selected. Note that Fig. 4B and Fig. 4C show the brightness range display section 50 within the window when the brightness range B and C are selected, respectively.

[0067] The brightness range display section 50 has a brightness range bar 51 , an upper limit indicator 52 , and a lower limit indicator 53 .

[0068] The luminance range bar 51 indicates the correspondence between the pixel values ​​of the constituent pixels of the second image data and the colors of the pixels that constitute the sample image 60 that is displayed in the window by the process of S106 described below.

[0069] As described above, the second image data is image data in which the output gradation values ​​of LUTs 13-1, 13-2, and 13-3 are used as pixel values ​​of the constituent pixels. The up and down directions of the brightness range bar 51 indicate the magnitude relationship of the output gradation values. The upper limit indicator 52 indicates the up and down position of the brightness range bar 51 corresponding to the maximum output gradation value, and the lower limit indicator 53 indicates the up and down position of the brightness range bar 51 corresponding to the minimum output gradation value.

[0070] 4A where the luminance range A is selected, the upper limit indicator 52 indicates the position of the luminance range bar 51 that corresponds to the output gradation value "255," and the lower limit indicator 53 indicates the position of the luminance range bar 51 that corresponds to the output gradation value "0." These values ​​correspond to the upper limit (when V=255) and the lower limit of the output gradation value in the graph G1 of FIG. 2, which shows the relationship between the intensity gradation value and the output gradation value for the LUT13-1 that corresponds to the luminance range A.

[0071] In the display examples of Figures 4A, 4B, and 4C, the area of ​​the brightness range bar 51 above the position indicated by the upper limit indicator 52 is displayed in a single color (e.g., light blue). The area from the position indicated by the upper limit indicator 52 to the position indicated by the lower limit indicator 53 is displayed as a gradation that changes continuously from the single color gradually to black. In each figure, this gradation is conveniently represented by a gradual change in the density of the shading. In this way, the output gradation value is represented by the color represented by the vertical position of the brightness range bar 51.

[0072] In S106, the above-mentioned display process is performed, and the sample image 60 represented by the second image data generated by the conversion process in S105 is displayed on the screen of the display device 20. This display process displays each component of the sample image 60 in the window in the color of the luminance range bar 51 corresponding to the output gradation value, which is the pixel value. In other words, the luminance range bar 51 represents the correspondence between the color of each component pixel of the sample image 60 and the output gradation value.

[0073] In S107, a process is performed to determine whether or not a new instruction to select a luminance range has been acquired as a result of a click operation on the luminance range selection button 41 by an operation on the input device 30. In this determination process, when it is determined that a new instruction to select a luminance range has been acquired (when the determination result is YES), the process proceeds to S108. On the other hand, in this determination process, when it is determined that a new selection instruction has not been acquired (when the determination result is NO), the process returns to S103, and the above-mentioned processes from S103 onwards are performed again. In the conversion process of S105 performed at this time, the same LUT as that used in the conversion process most recently executed is used among LUTs 13-1, 13-2, and 13-3.

[0074] In S108, a process is performed in which each icon button of the brightness range selection button 41 displayed on the screen of the display device 20 is changed to a display mode corresponding to the brightness range selection instruction determined to have been acquired in the determination process of S107. After that, the process returns to S103, and the above-mentioned processes from S103 onwards are performed again. In the conversion process of S105 performed at this time, one of the LUTs 13-1, 13-2, and 13-3 that corresponds to the brightness range instructed by the brightness range selection instruction determined to have been acquired in the determination process of S107 is used.

[0075] 4A, 4B, and 4C are displayed on the display device 20 as a result of the processor 11 performing the above processes. When the user of the image processing system 1 operates the input device 30 to click on the brightness range selection button 41, the brightness range is switched and the selection of the LUTs 13-1, 13-2, and 13-3 is changed and is immediately reflected in the display of the sample image 60 on the screen. Therefore, the sample image 60 can be quickly adjusted.

[0076] It is possible for the display mode of the image that the processor 11 causes the display device 20 to display based on the second image data to be different from the display modes exemplified in Figures 4A, 4B, and 4C. As an example of such a different display mode, Figure 5 shows a second example of a display screen on the display device 20, and both Figures 6A and 6B show a third example of a display screen on the display device 20.

[0077] First, a second example of the display screen will be described. The example screen of Fig. 5 is a screen that is displayed when the display screen of Fig. 4A is displayed and the Hi-Lo button 42 included in the icon button group 40 is clicked by operating the input device 30 to put the icon button into a pressed state.

[0078] In the example screen of FIG. 5, the display mode of the luminance range bar 51 is different from that in FIG. 4A. More specifically, the area above the position indicated by the upper limit indicator 52 in the luminance range bar 51 (the area with diagonal lines in a direction from the lower left to the right in FIG. 5) is displayed uniformly in a first color (e.g., red). The position indicated by the upper limit indicator 52 may be included in the area displayed in the first color. The area corresponding to the position indicated by the lower limit indicator 53 is displayed in a second color (e.g., dark blue) different from the first color. The area from the position indicated by the upper limit indicator 52 to the position of the second color indicated by the lower limit indicator 53 is displayed as a gradation that changes continuously from a color different from both the first and second colors, such as white, to black. In FIG. 5, this gradation is conveniently expressed by a uniform shading.

[0079] In addition, when using an LUT in which a relationship is set such that an output gradation value of 0 corresponds to an intensity gradation value close to 0, as described above, the area below the position indicated by the lower limit indicator 53 is also displayed in the second color.

[0080] In the process of S103 in the process of Fig. 3 described above, processor 11 determines whether Hi-Lo button 42 is in a pressed state. If it is determined that Hi-Lo button 42 is not in a pressed state, processor 11 performs a process of displaying brightness range display section 50 illustrated in Fig. 4A on the screen of display device 20. On the other hand, if it is determined that Hi-Lo button 42 is in a pressed state, processor 11 performs a process of displaying brightness range display section 50 illustrated in Fig. 5 on the screen of display device 20.

[0081] In the process of S106 performed after the above-mentioned process, the display process is performed as described above, and the sample image 60 represented by the second image data generated by the conversion process of S105 is displayed on the screen of the display device 20. By this display process, each component of the sample image 60 in the window is displayed in the color of the luminance range bar 51 corresponding to the output gradation value, which is the pixel value. Therefore, when the luminance range bar 51 is in the display mode shown in FIG. 5, in the sample image 60, an area (saturation area 61 in FIG. 5) consisting of component pixels whose pixel values ​​exceed the output gradation value indicated by the upper limit indicator 52 is uniformly displayed in the first color described above. Also, in the sample image 60, an area (background area 62 in FIG. 5) whose pixel values ​​are the output gradation value indicated by the lower limit indicator 53 is uniformly displayed in the second color described above.

[0082] In this way, when Hi-Lo button 42 is pressed down by a click operation, processor 11, which has acquired this click operation, performs a process of uniformly changing the constituent pixels of the second image data corresponding to the constituent pixels of the first image data whose pixel values ​​exceed a predetermined upper limit to a first color. At this time, processor 11 also performs a process of uniformly changing the constituent pixels of the second image data corresponding to the constituent pixel in the first image data whose pixel value is the smallest to a second color different from the first color.

[0083] Furthermore, at this time, the processor 11 performs processing to set the remaining constituent pixels of the second image data to the color corresponding to the output gradation value, which is the pixel value, i.e., the color indicated by the brightness range bar 51 which indicates the correspondence between the colors of the constituent pixels and the output gradation values. Then, the processor 11 performs processing to cause the display device 20 to display the sample image 60 in the color thus set for each constituent pixel.

[0084] By displaying the second example of the display screen obtained as described above, it is possible to instantly know whether or not an area exceeding the upper limit value or the lower limit value set for the output gradation value exists in the sample image 60. Therefore, for example, in the above-mentioned image search (sample search) work, it becomes possible to instantly determine whether or not there is a possibility that laser light of excessive irradiation intensity has been applied to the sample.

[0085] In addition, when the Hi-Lo button 42 is clicked on the display screen of FIG. 4B or FIG. 4C and the icon button is put into a pressed state, the same display process is performed and the brightness range bar 51 and sample image 60 in the same display mode as in FIG. 5 are displayed on the screen.

[0086] Next, a third example of the display screen will be described. The screen examples of Figures 6A and 6B are screens that are displayed when the display screen of Figure 4B is displayed and the Hi-Lo button 42 included in the icon button group 40 is clicked by operating the input device 30 to put the icon button in a pressed state.

[0087] In the example screens of Fig. 6A and Fig. 6B, the display mode of the brightness range bar 51 is different from that in Fig. 4B. More specifically, the area above the position indicated by the upper limit indicator 52 in the brightness range bar 51 (the area shaded from the upper left to the lower right in Fig. 6A and Fig. 6B) is displayed uniformly in a first color (e.g., orange). The position indicated by the upper limit indicator 52 may be included in the area displayed in the first color. Also, the area corresponding to the position indicated by the lower limit indicator 53 is displayed in a second color (e.g., dark blue) different from the first color.

[0088] In addition, when using an LUT in which a relationship is set such that an output gradation value of 0 corresponds to an intensity gradation value close to 0, as described above, the area below the position indicated by the lower limit indicator 53 is also displayed in the second color.

[0089] 6A and 6B, the region from the position indicated by the upper limit indicator 52 to the position of the second color indicated by the lower limit indicator 53 is displayed as a gradation that changes continuously from a color different from either the first or second color, such as white, to black. Note that in FIGS. 6A and 6B, this gradation is conveniently represented by uniform shading. Furthermore, the region that defines the upper end of the brightness range bar 51, that is, the region at the upper end of the region displayed in the first color (the region with diagonal lines extending from the lower left to the upper right), is displayed in a third color (e.g., red) that is different from the color used in the gradation, the first color, and the second color.

[0090] 3, the processor 11 determines whether the Hi-Lo button 42 is pressed down. If it is determined that the Hi-Lo button 42 is not pressed down, the processor 11 performs a process of displaying the brightness range display section 50 illustrated in FIG. 4B on the screen of the display device 20. On the other hand, if it is determined that the Hi-Lo button 42 is pressed down, the processor 11 performs a process of displaying the brightness range display section 50 illustrated in FIG. 6A and FIG. 6B on the screen of the display device 20.

[0091] In the process of S106 performed after the above-mentioned process, the display process is performed as described above, and the sample image 60 represented by the second image data generated by the conversion process of S105 is displayed on the screen of the display device 20. By this display process, each component of the sample image 60 in the window is displayed in the color of the luminance range bar 51 corresponding to the output gradation value, which is the pixel value. Therefore, when the luminance range bar 51 is in the display mode shown in FIG. 6A, in the sample image 60, an area (first saturated area 61-1 in FIG. 6A) consisting of component pixels whose pixel values ​​are greater than the output gradation value indicated by the upper limit indicator 52 is uniformly displayed in the first color. Also, in the sample image 60, an area (background area 62 in FIG. 6A) whose pixel values ​​are the output gradation value indicated by the lower limit indicator 53 is uniformly displayed in the above-mentioned second color.

[0092] In addition, in the process of S106, the processor 11 further performs a process of displaying, in the above-mentioned third color, the constituent pixels of the sample image 60 corresponding to the constituent pixels of the first image data whose pixel values, which are intensity gradation values, cannot be expressed in 16-bit length and have overflowed. For this purpose, the computer 10 further acquires from the optical device 100 an overflow detection signal output from an AD converter 113 that samples the intensity signal output from the PPD 111. The processor 11 identifies the target constituent pixel in the sample image 60 based on this overflow detection signal and the scanning position of the scanner corresponding to the detection signal, and performs a process of displaying the identified constituent pixel as the third color.

[0093] FIG. 6B shows an example of a screen when the output of the laser 107 is increased to increase the irradiation intensity of the laser light onto the sample in a state in which the sample image 60 of FIG. 6A has been obtained.

[0094] In FIG. 6B, a region of the sample image 60 consisting of constituent pixels whose output gradation values ​​were expressed in colors within the gradation range from white to black in the brightness range bar 51 in the screen example of FIG. 6A is represented as a first saturated region 61-1. This indicates that the region has become a region consisting of constituent pixels whose output gradation values ​​are greater than the output gradation value indicated by the upper limit indicator 52 by increasing the irradiation intensity of the laser light on the sample. Also, in FIG. 6B, a second saturated region 61-2 appears, whose constituent pixels are expressed in a third color. This second saturated region 61-2 is a region of constituent pixels that was represented as the first saturated region 61-1 in FIG. 6A, and indicates that the intensity gradation values ​​of the constituent pixels of the region have overflowed by increasing the irradiation intensity of the laser light on the sample.

[0095] In this way, when Hi-Lo button 42 is pressed down by a click operation, processor 11, which has acquired this click operation, performs a process of uniformly changing the constituent pixels of the second image data corresponding to the constituent pixels of the first image data whose pixel values ​​exceed a predetermined upper limit to a first color. At this time, processor 11 also performs a process of uniformly changing the constituent pixels of the second image data corresponding to the constituent pixel in the first image data whose pixel value is the smallest to a second color different from the first color.

[0096] Furthermore, at this time, the processor 11 performs a process of acquiring from the optical device 100 a flag indicating that an intensity signal indicating a light intensity exceeding a predetermined maximum value has been output from the PPD 11. This flag is, for example, a flag indicating the occurrence of an overflow in AD conversion, which is output from the AD converter 113 included in the optical device 100. In response to acquiring this flag, the processor 11 further performs a process of uniformly changing the constituent pixels of the second image data corresponding to the constituent pixels in the first image data whose pixel values ​​indicate a light intensity exceeding the maximum value to a third color different from both the first and second colors.

[0097] Furthermore, at this time, the processor 11 performs processing to set the remaining constituent pixels of the second image data to a color corresponding to the output gradation value, which is the pixel value (for example, a gradation of a color different from any of the first, second, and third colors), that is, the color indicated by the brightness range bar 51 indicating the correspondence between the color of the constituent pixels and the output gradation value. Then, the processor 11 performs processing to cause the display device 20 to display the sample image 60 in the color thus set for each constituent pixel.

[0098] By displaying the third example of the display screen obtained as described above, it becomes possible to quickly and smoothly carry out adjustment work to keep the irradiation intensity of the laser light on the sample during normal observation within an appropriate range.

[0099] In addition, when the Hi-Lo button 42 is clicked on the display screen of FIG. 4A or FIG. 4C and the icon button is put into a pressed state, the same display process is performed, and the brightness range bar 51 and sample image 60 are displayed on the screen in the same manner as in FIG. 6A and FIG. 6B.

[0100] The above-described embodiment is a specific example for facilitating understanding of the invention, and the embodiment of the present invention is not limited to this. Various modifications and changes can be made to the image processing system 1 without departing from the scope of the claims.

[0101] For example, in the above-described embodiment, an example has been shown in which three LUTs 13-1, 13-2, and 13-3 are stored in the memory 12, but the number of LUTs may be any number as long as it is plural. [Explanation of symbols]

[0102] 1. Image Processing System 10. Computers 11 Processors 12. Memory 13-1, 13-2, 13-3 LUTs 20 Display device 30 Input Devices 40 Icon Buttons 41 Luminance range selection button 42 Hi-Lo button 50 Luminance range display 51 Luminance Range Bar 52 Upper limit indicator 53 Lower limit indicator 60 Sample Images 61 Saturation region 61-1 1st saturation region 61-2 2nd saturation region 62 Background area 100 Optical equipment 105 Scan Unit 107 Laser 109 Confocal Aperture 111 PPD

Claims

1. A scanner and PPD (Pixelated Photon Detector) and a memory storing a plurality of tables in which intensity gradation values ​​correspond to output gradation values; 1. A processor comprising: generating first image data in which the intensity gradation value indicated by the light intensity signal is set as a pixel value of a constituent pixel based on the light intensity signal output from the PPD and the scanning position of the scanner; converting the first image data based on one of the tables stored in the memory, to generate second image data in which the output gradation values ​​are pixel values ​​of component pixels; displaying an image represented by the second image data on a display unit; The processor is configured to perform processing; An image processing system comprising:

2. The processor is configured to perform a process of obtaining an input for selecting one of the tables; the second image data is generated by converting the first image data based on the one of the tables selected by the input; 2. The image processing system according to claim 1.

3. The image processing system of claim 1 or 2, characterized in that the processor is configured to perform processing to display on the display unit an image in which the colors of the constituent pixels of the second image data corresponding to the constituent pixels of the first image data whose pixel values ​​exceed a predetermined upper limit value, or the colors of the constituent pixels of the second image data corresponding to the constituent pixels of the first image data whose pixel values ​​are equal to or greater than the predetermined upper limit value, are displayed as a first color different from the color used to display pixels below the predetermined upper limit value or pixels that do not exceed the predetermined upper limit value.

4. The image processing system of claim 3, wherein the processor is configured to perform processing to display on the display unit an image in which constituent pixels of the second image data corresponding to constituent pixels in the first image data whose pixel values ​​are below a predetermined lower limit value are colored a second color different from the first color and different from the color of pixels that do not exceed the predetermined upper limit value.

5. The processor, obtaining a flag indicating that the light intensity signal indicating the intensity gradation value exceeding a predetermined maximum value has been output from the PPD; In response to acquisition of the flag, display the image on the display unit in which a pixel of the second image data corresponding to a pixel in the first image data that expresses the intensity gradation value whose pixel value exceeds the maximum value is displayed in a third color different from a color that displays a pixel whose pixel value does not exceed a predetermined upper limit value.

3. An image processing system according to claim 1, characterized in that the image processing system is configured to perform processing.

6. The processor, obtaining a flag indicating that the light intensity signal indicating the intensity gradation value exceeding a predetermined maximum value has been output from the PPD; In response to acquisition of the flag, display the image on the display unit by changing a constituent pixel of the second image data corresponding to a constituent pixel in the first image data that expresses the intensity gradation value exceeding the maximum value to a third color that is different from both the first color and the second color and is also different from a color of a pixel that does not exceed the predetermined upper limit value.

5. An image processing system according to claim 4, configured to perform processing.

7. The processor, displaying the image on the display unit, the pixels of the second image data being colored in a manner corresponding to the pixel values ​​of the pixels; displaying on the display unit a display showing a correspondence relationship between the color and the output gradation value for each pixel constituting the image; 7. An image processing system according to claim 6, configured to perform processing.

8. generating first image data in which the intensity gradation value indicated by the light intensity signal is set as the pixel value of each of the constituent pixels based on a light intensity signal output from a pixelated photon detector (PPD) and a scanning position of the scanner; converting the first image data based on one selected from a table in which the intensity gradation value corresponds to an output gradation value and a plurality of the tables are stored in a memory, to generate second image data in which the output gradation value is a pixel value of a constituent pixel; displaying an image represented by the second image data on a display unit; 13. An image processing method comprising:

Images