Information processing device, information processing system, information processing method, and information processing program

Abstract

To quickly execute automatic focusing.SOLUTION: An information processing apparatus 4 is configured to: acquire pieces of image data obtained by capturing an image of a target object to be observed by using a microscope 2 at respective positions where an object lens 21 of the microscope 2 is driven, in an optical-axis direction of illumination with which the target object is irradiated, the image being obtained by condensing light by using the object lens 21; specify a focusing position of the object lens 21 at which a focus of the target object is determined based on the pieces of image data; in a case where the object lens 21 is arranged in the focusing position, acquire, as a reference light receiving pattern, a light receiving pattern obtained by receiving, via the object lens 21, focus detecting light (reflected light) 228 emitted to the target object via the object lens 21; and store the reference light receiving pattern in a storage.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing system, an information processing method, and an information processing program.

Background Art

[0002] Conventionally, in imaging a sample using a microscope, there is a technique for determining the focus of an objective lens based on the size and position of a focus detection light spot that changes by driving the objective lens of the microscope in the optical axis direction.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technology, it is difficult to perform autofocus at high speed. For example, the above conventional technology assumes that the bottom surface of the container is flat and uniform. In the case of a cell container subjected to three-dimensional culture, the focus detection light does not reflect normally, and the spot of the focus detection light acquired by the imaging device becomes deformed, and it may become impossible to easily determine the focus.

[0005] The present invention has been made in view of the above, and an object thereof is to perform autofocus at high speed.

Means for Solving the Problems

[0006] The present invention provides an information processing device comprising: a first acquisition unit that acquires image data of an object focused by an objective lens at each position where the objective lens of the microscope is driven in the optical axis direction when illumination is shone on an object to be observed using a microscope; a first identification unit that identifies the focus position of the objective lens where the object is in focus based on the image data; and a second acquisition unit that, when the objective lens is set to the focus position, acquires a light reception pattern as a reference light reception pattern obtained by receiving the reflected light of the focusing light shone on the object through the objective lens via the objective lens, and stores the reference light reception pattern in a storage unit.

[0007] The present invention provides an information processing system comprising an imaging device, a microscope, and an information processing device, wherein the imaging device captures an image of the object focused by the objective lens at each position where the objective lens of the microscope is driven in the direction of the optical axis when illumination is shone on the object to be observed using the microscope, the microscope has a light-emitting device that irradiates the object with focusing light through the objective lens, and the information processing device includes a first acquisition unit that acquires image data of the image of the object captured by the imaging device, a first identification unit that identifies the focus position of the objective lens where the object is in focus based on the image data, and a second acquisition unit that, when the objective lens is set to the focus position, acquires a light-receiving pattern as a reference light-receiving pattern obtained by receiving the reflected light of the focusing light irradiated on the object by the light-emitting device through the objective lens, and stores the reference light-receiving pattern in a storage unit.

[0008] Furthermore, the present invention provides an information processing method in which a computer acquires image data of the object focused by the objective lens at each position where the objective lens of the microscope is driven in the direction of the optical axis when illumination is shone on the object to be observed using the microscope, identifies the focus position of the objective lens where the object is in focus based on the image data, acquires the light reception pattern obtained by receiving the reflected light of the focusing light shone on the object through the objective lens when the objective lens is set to the focus position, and stores the reference light reception pattern in a memory unit.

[0009] Furthermore, the present invention provides an information processing program that causes a computer to perform the following processes: acquire image data of the object focused by the objective lens at each position where the objective lens of the microscope is driven in the direction of the optical axis when illumination is shone on the object to be observed using the microscope; identify the focus position of the objective lens where the object is in focus based on the image data; acquire the light reception pattern obtained by receiving the reflected light of the focusing light shone on the object through the objective lens when the objective lens is set to the focus position as a reference light reception pattern; and store the reference light reception pattern in a memory unit. [Effects of the Invention]

[0010] According to the present invention, there is an effect that autofocus can be performed at high speed. [Brief explanation of the drawing]

[0011] [Figure 1] This figure shows an example configuration of an information processing system according to the embodiment. [Figure 2] This figure shows an example of a light reception pattern 1 of the focusing light according to the embodiment. [Figure 3] This figure shows an example of a light reception pattern 2 for the focusing light according to the embodiment. [Figure 4] This figure shows an example of a reference light reception pattern identification process according to the embodiment. [Figure 5] This block diagram shows an example of the configuration of each device in the information processing system according to the embodiment. [Figure 6] This flowchart shows an example of the flow of the reference light reception pattern identification process according to the embodiment. [Figure 7] This flowchart shows an example of the sample imaging process according to the embodiment. [Figure 8] This is a diagram illustrating an example hardware configuration. [Modes for carrying out the invention]

[0012] An information processing apparatus, information processing system, information processing method, and information processing program according to one embodiment of the present invention will be described in detail below with reference to the drawings. However, the present invention is not limited to the embodiments described below.

[0013] [Embodiment] The configuration of the information processing system according to the embodiment, the configuration of the information processing device, and the flow of each process will be described in order below, and finally the effects of the embodiment will be described.

[0014] [1. Configuration of Information Processing System 100] The configuration of the information processing system 100 according to the embodiment will be described in detail using Figure 1. Figure 1 is a diagram showing an example of the configuration of the information processing system 100 according to the embodiment. In the following, the overall configuration example of the information processing system 100, the processing of the information processing system 100, and the effects of the information processing system 100 will be described in that order.

[0015] (1-1. Example of the overall configuration of the information processing system 100) The information processing system 100 includes a container 1, a microscope 2, an imaging device (camera 3), and an information processing device 4. The information processing system 100 shown in Figure 1 may include multiple containers 1, multiple microscopes 2, multiple cameras 3, or multiple information processing devices 4. Furthermore, the information processing device 4 may be integrated with one or more of the microscopes 2 and cameras 3. The container 1, microscopes 2, cameras 3, and information processing device 4 will be described below.

[0016] (1-1-1. Container 1) Container 1 has multiple compartments 11 for containing samples such as cultured cells to be observed by microscope 2. In the following description, the multiple compartments 11 may be referred to as the first compartment 11-1, the second compartment 11-2, and so on. Container 1 is also placed on a stand (not shown) so that the samples can be observed by microscope 2.

[0017] (1-1-2. Microscope 2) Microscope 2 comprises an objective lens 21, a focusing device 22, a drive device 23, and an imaging lens 24. Here, the objective lens 21 is installed between the container 1 and the focusing device 22 in the optical axis direction (or, as appropriate, the "Z direction"), and its position can be changed by the drive device 23. The focusing device 22 comprises a light-emitting device 221, a first lens 222, a first spectroscopic element 223, a second spectroscopic element 224, a second lens 225, and a two-dimensional optical sensor 226. The drive device 23 is controllably connected to the information processing device 4 and drives the objective lens 21 in the Z direction based on signals from the information processing device 4. The imaging lens 24 is installed between the focusing device 22 and the camera 3 in the Z direction.

[0018] (1-1-3. Camera 3) Camera 3 is positioned to receive observation light 5 emitted from the sample through the microscope 2. Here, camera 3 captures an image of the received light based on a signal from the information processing device 4 and transmits the image data of the received light to the information processing device 4.

[0019] (1-1-4. Information Processing Device 4) The information processing device 4 is connected to the microscope 2 and controls the drive process of the drive device 23 of the microscope 2. The information processing device 4 is also connected to the microscope 2 and acquires the pattern of the focused light (reflected light) 228 (referred to as "receiving pattern" as appropriate) via the two-dimensional optical sensor 226 of the microscope 2. The information processing device 4 is also connected to the camera 3 and controls the imaging process of the camera 3 to acquire image data of the captured sample.

[0020] (1-2. Overall processing of the information processing system 100) The processing of the entire information processing system 100 described above will now be explained. Note that the following processes can be executed in a different order. Also, some of the following processes may be omitted.

[0021] (1-2-1. Light reception pattern acquisition process) The following describes the process of acquiring the light reception pattern of the focused light (reflected light) 228 from container 1, using Figure 1.

[0022] The focusing device 22 emits focusing light by illuminating the light-emitting device 221. At this time, the focusing light (irradiated light) 227 emitted from the light-emitting device 221 is converted into parallel light by the first lens 222, reflected by the first spectroscopic element 223, reflected by the second spectroscopic element 224, focused by the objective lens 21, and guided to the container 1. The focusing light (reflected light) 228 reflected from the bottom surface of the container 1 passes through the objective lens 21 again, is reflected by the second spectroscopic element 224, passes through the first spectroscopic element 223, is focused by the second lens 225, and is received by the two-dimensional optical sensor 226. The information processing device 4 acquires the received light pattern from the two-dimensional optical sensor 226 as image data.

[0023] Here, the drive device 23 changes the Z-direction position of the objective lens 21 (referred to as "Z position" as appropriate), and the above process is executed. As a result, the state of the focused light (reflected light) 228 changes due to the difference in the Z position of the objective lens 21, or in other words, the difference in the illumination state of the focused light (irradiated light) 227 to the container 1, and the light reception pattern projected onto the two-dimensional optical sensor 226 changes. At this time, if the bottom surface of the container 1 is flat (planar) as shown in Figure 2(1), the light reception pattern of the focused light (reflected light) 228 is regular, and focus detection is easy. However, if the bottom surface of the container 1 is curved as shown in Figure 3(1), the light reception pattern of the focused light (reflected light) 228 becomes irregular, making it difficult to perform focus detection using only this light reception pattern.

[0024] (1-2-2. Light Reception Pattern) Here, we will explain the light reception patterns acquired by the information processing device 4 using Figures 2 and 3. Below, we will explain light reception pattern 1 when the bottom surface of container 1 is flat, and light reception pattern 2 when the bottom surface of container 1 is curved.

[0025] (1-2-2-1. Light Reception Pattern 1) The light receiving pattern 1 when the bottom surface of container 1 is flat will be explained using Figure 2. Figure 2 is a diagram showing an example of the light receiving pattern 1 of the focusing light according to the embodiment.

[0026] Using Figure 2(1), the relationship between the container compartment 11 and the reflection position of the detection light (irradiation light) 227 due to the difference in the Z position of the objective lens 21 will be explained. Here, Z0, the Z position of the objective lens 21, is the position where the focal point coincides with the bottom surface of the container compartment 11. As shown in the example in Figure 2(1), when the objective lens 21 is at Z0, the detection light (irradiation light) 227 is reflected from the bottom surface of the container compartment 11. +1 This is the position where the objective lens 21 is brought closer to the container 1 in the Z direction. As shown in the example in Figure 2(1), when the objective lens 21 is Z +1In this case, the focusing light (irradiation light) 227 is reflected at a position away from the bottom surface of the container compartment 11 in the Z direction relative to the objective lens 21. Also, the Z position of the objective lens 21 is Z -1 This is the position where the objective lens 21 is moved away from the container 1 in the Z direction. As shown in the example in Figure 2(1), the objective lens 21 is Z -1 In this case, the focusing light (irradiation light) 227 is reflected at a position closer to the objective lens 21 in the Z direction than the bottom surface of the container compartment 11.

[0027] Using Figure 2(2), the light reception patterns of the focusing light (reflected light) 228 projected onto the two-dimensional optical sensor 226, depending on the Z position of the objective lens 21, will be explained. As shown in the example in Figure 2(2), when the objective lens 21 is at Z0, the light reception pattern of the focusing light (reflected light) 228 is obtained as a clear and minimal spot. On the other hand, as shown in the example in Figure 2(2), when the objective lens 21 is at Z +1 , Z -1 In this case, the light reception pattern of the focusing light (reflected light) 228 is obtained as an indistinct spot. That is, if the bottom surface of container 1 is flat as shown in Figure 2(1), the light reception pattern of the focusing light (reflected light) 228 is regular, and focus detection is easy.

[0028] (1-2-2-2. Light Reception Pattern 2) Using Figure 3, we will explain the light receiving pattern 2 when the bottom surface of container 1 is curved. Figure 3 is a diagram showing an example of the light receiving pattern 2 of the focusing light according to the embodiment.

[0029] Using Figure 3(1), the relationship between the container compartment 11 and the reflection position of the detection light (irradiation light) 227 due to the difference in the Z position of the objective lens 21 will be explained. Here, Z0, the Z position of the objective lens 21, is the position where the focal point coincides with the bottom surface of the container compartment 11. As shown in the example in Figure 3(1), when the objective lens 21 is at Z0, the detection light (irradiation light) 227 is reflected from the bottom surface of the container compartment 11. +1 This is the position where the objective lens 21 is brought closer to the container 1 in the Z direction. As shown in the example in Figure 3(1), when the objective lens 21 is Z+1 In the case of, the focusing light (irradiation light) 227 is reflected at a position farther away from the objective lens 21 in the Z direction than the bottom surface of the partition 11 of the container. Also, the Z position of the objective lens 21, Z -1 is a position where the objective lens 21 is moved farther away from the container 1 in the Z direction. As shown in the example of Fig. 3(1), when the objective lens 21 is at Z -1 In the case of, the focusing light (irradiation light) 227 is reflected at a position closer to the objective lens 21 in the Z direction than the bottom surface of the partition 11 of the container.

[0030] Using Fig. 3(2), the light reception pattern of the focusing light (reflected light) 228 due to the difference in the Z position of the objective lens 21 projected onto the two-dimensional optical sensor 226 will be described. As shown in the example of Fig. 3(2), regardless of the Z positions of the objective lens 21 being Z0, Z +1 , Z -1 , the light reception pattern of the focusing light (reflected light) 228 obtained by the objective lens 21 is an unclear spot. That is, when the bottom surface of the container 1 is a curved surface as shown in Fig. 3(1), the light reception pattern of the focusing light (reflected light) 228 becomes irregular, so it is difficult to perform focus detection only based on this light reception pattern.

[0031] (1-2-3. Reference light reception pattern acquisition process) Here, using Fig. 4, when the bottom surface of the container 1 is a curved surface and the sample 12 is accommodated in the partition 11, the process of acquiring the reference pattern (reference light reception pattern) among the light reception patterns will be described. Fig. 4 is a diagram showing an example of the reference light reception pattern identification process according to the embodiment. Below, after explaining the reflection position of the focusing light (irradiation light) 227, the light reception pattern of the focusing light (reflected light) 228, and the image data of the sample 12, the reference light reception pattern acquisition process will be described.

[0032] (1-2-3-1. Reflection position of the focusing light (irradiation light) 227) Using Figure 4(1), the relationship between the compartment 11 of the container containing the sample 12 and the reflection position of the detection light (irradiation light) 227 due to the difference in the Z position of the objective lens 21 will be explained. Here, Z0, the Z position of the objective lens 21, is the position where the focal point matches that of the sample 12 in the compartment 11 of the container. As shown in the example in Figure 4(1), when the objective lens 21 is at Z0, the detection light (irradiation light) 227 is reflected by the sample 12 in the compartment 11 of the container. +1 This is the position where the objective lens 21 is brought closer to the container 1 in the Z direction. As shown in the example in Figure 4(1), when the objective lens 21 is Z +1 In this case, the focusing light (irradiation light) 227 is reflected at a position in the Z direction away from the objective lens 21 from the sample 12 in the container compartment 11. Also, the Z position of the objective lens 21 is Z -1 This is the position where the objective lens 21 is moved away from the container 1 in the Z direction. As shown in the example in Figure 4(1), the objective lens 21 is Z -1 In this case, the focusing light (irradiation light) 227 is reflected from a position closer to the objective lens 21 in the Z direction than the sample 12 in compartment 11 of the container.

[0033] (1-2-3-2. Light reception pattern of the focused light (reflected light) 228) Using Figure 4(2), the light reception pattern of the focusing light (reflected light) 228 projected onto the 2D optical sensor 226 due to differences in the Z position of the objective lens 21 will be explained. As shown in the example in Figure 4(2), the objective lens 21 is Z0, Z +1 , Z -1 Regardless of the Z position, the light reception pattern of the focusing light (reflected light) 228 by the objective lens 21 is obtained as an indistinct spot. That is, if the bottom surface of the container 1 is curved as shown in Figure 4(1), the light reception pattern of the focusing light (reflected light) 228 becomes irregular, similar to Figure 3(1) described above, making it difficult to detect focus using only this light reception pattern.

[0034] (1-2-3-3. Image data of sample 12) Using Figure 4(3), we will explain the image data of the sample 12 captured by the camera 3, showing the difference in the Z position of the objective lens 21. As shown in the example in Figure 4(3), when the objective lens 21 is at Z0, the sample 12 is captured as a clear image. On the other hand, as shown in the example in Figure 4(3), when the objective lens 21 is at Z0, +1 , Z -1 In this case, sample 12 is captured as an unclear image.

[0035] (1-2-3-4. Acquisition process of reference light reception pattern) Using Figure 1, the process for acquiring a reference light reception pattern will be explained when the bottom surface of container 1 is curved, as shown in the example in Figure 4, and the sample 12 is contained in compartment 11. Details of the reference light reception pattern acquisition process will be described later in [3. Flow of Information Processing System 100] (3-1. Flow of Reference Light Reception Pattern Acquisition Process).

[0036] First, the illumination device (not shown) illuminates the sample 12 contained in compartment 11 of container 1. At this time, the observation light 5 emitted from the sample 12 due to the illumination passes through the objective lens 21, passes through the second spectroscopic element 224, is focused by the imaging lens 24, and enters the camera 3.

[0037] Next, the drive unit 23 of the microscope 2 drives the objective lens 21 in response to a signal from the information processing device 4, changing the Z position of the objective lens 21. The camera 3 also repeatedly performs imaging and transfers the captured image data to the information processing device 4. At this time, the information processing device 4 links the transferred image data with the Z position of the objective lens 21.

[0038] Next, the information processing device 4 analyzes the acquired image data, selects the image data that is best in focus on the sample 12, and sets the Z position of the objective lens 21 corresponding to that image data as the focus position Z f In this case, the information processing device 4 uses the feature quantities of the image data (e.g., brightness value, contrast value, etc.) to determine the focus position Z f Identify.

[0039] Then, after the illumination device (not shown) is turned off, the drive unit 23 of the microscope 2 drives the objective lens 21 in response to the signal from the information processing device 4, and focuses to position Z f Move to [location].

[0040] Finally, the information processing device 4 acquires the light reception pattern of the focusing light (reflected light) 228 returned from the container 1 by the light emission of the light emission device 221 of the microscope 2, and uses this light reception pattern as the reference pattern for when the light is focused, i.e., the reference light reception pattern. At this time, the information processing device 4 uses the acquired reference light reception pattern and the focus position Z f It links and remembers.

[0041] The information processing device 4 may perform the above-described reference light reception pattern acquisition process each time it images the container 1. Alternatively, the information processing device 4 may pre-process the reference light reception pattern acquisition process for each combination of objective lens 21 and container 1, store the reference light reception pattern for each combination in the storage unit 44 of the information processing device 4 (described later), and recall and use it as needed during the sample imaging process.

[0042] (1-2-4. Sample imaging and processing) Using Figure 1, the process of imaging a sample based on the reference light-receiving pattern obtained through the above-described reference light-receiving pattern acquisition process will be explained. Details of the sample imaging process will be described later in [3. Flow of Information Processing System 100] (3-2. Flow of Sample Imaging Process).

[0043] First, the information processing device 4 acquires the light reception pattern of the focusing light (reflected light) 228 returned from the container 1 by the light emission of the light emission device 221 of the microscope 2, which is detected by the two-dimensional optical sensor 226. At this time, the information processing device 4 acquires multiple light reception patterns while driving the objective lens 21 in the Z direction by transmitting a signal to the drive device 23 of the microscope 2.

[0044] Next, the information processing device 4 identifies a light-receiving pattern that matches the reference light-receiving pattern, and sets the Z position where the light-receiving pattern is obtained as the focus position (imaging reference position) Z.f1 In this case, the information processing device 4 may perform the matching determination of the light reception patterns using the cross-correlation value of the images, or it may perform the determination by calculating the similarity using a machine learning model.

[0045] Then, after the light-emitting device 221 is turned off, the illumination device (not shown) illuminates the sample 12 contained in compartment 11 of container 1. In addition, the drive device 23 of the microscope 2 drives the objective lens 21 by the signal from the information processing device 4, and focuses to position Z f1 Move to the next position. Finally, camera 3 will be in focus position Z. f1 Using this as the reference Z position, the sample 12 is imaged, and the captured image data is transferred to the information processing device 4.

[0046] (1-3. Effects of Information Processing System 100) Below, as reference technology, we will explain the problems with the technologies commonly used for autofocus and image processing, and then describe the effects of the information processing system 100.

[0047] (1-3-1. Problems) In the reference technology described in Patent Document 1, the detection light emitted from the light source is converted into parallel light by the first lens, reflected by a beam splitter, focused by the objective lens, reflected by a plate, passes through the objective lens again, passes through the beam splitter, passes through the aperture, is focused by the second lens, and enters the imaging device. When the objective lens is driven in the direction of the optical axis, the position and size of the detection light spot imaged by the imaging device change. The focal point of the objective lens relative to the plate is determined based on the size and position of this detection light spot. The above technology has the following problems.

[0048] Firstly, in recent biological and pharmaceutical research, it has become important to perform cell experiments under conditions closer to those of living organisms by culturing cells in three dimensions rather than in a planar manner. Among the methods commonly used in this regard is the creation of cell aggregates. For example, Patent Document 2 describes the use of a cell container with a curved cell culture surface to facilitate the formation of cell aggregates. Furthermore, Patent Document 3 describes the use of a cell container with a three-dimensional structure formed on the cell culture surface to facilitate the formation of cell aggregates.

[0049] On the other hand, the autofocus technology in the reference technology of Patent Document 1 assumes that the bottom surface of the cell container is flat and uniform. In the case of cell containers with a curved cell culture surface (see, for example, Patent Document 2) or cell containers with a three-dimensional structure formed on the cell culture surface (see, for example, Patent Document 3), the focused light is not reflected properly, and the spot of the focused light acquired by the imaging device becomes irregular in shape, making it difficult to easily determine the focus. Specifically, even if the objective lens is in focus on the cell container, in the case of cell containers with a curved cell culture surface, the spot of the focused light in the imaging device becomes irregular in shape due to aberration of the focused light caused by the curved surface of the cell culture surface, or in the case of cell containers with a three-dimensional structure formed on the cell culture surface, due to scattering of the focused light caused by the three-dimensional structure of the cell culture surface, making it impossible to determine its size and position.

[0050] Another reference technique involves changing the position of the objective lens along its optical axis and capturing images of the target cells each time. The focus is then determined from the characteristic features of these images (such as brightness and contrast values). This method allows for the search for the focus regardless of the condition of the bottom surface of the cell container.

[0051] However, the above technology has the problem that, because a cell image must be captured at each objective lens position, the focus search time increases due to the exposure time for imaging, the data transfer time from the imaging device to the processing device, and the image analysis time. In addition, with the above technology, if the cell imaging method is epifluorescence or fluorescence confocal, excitation light is irradiated onto the cells before imaging, which can cause fading of the fluorescent dye and photodamage to the cells.

[0052] (1-3-2. Overview) The information processing system 100 acquires image data of the sample 12 focused by the objective lens 21 at each position where the objective lens 21 of the microscope 2 is driven in the direction of the optical axis when the sample 12 is illuminated using the microscope 2. Based on the image data, the system determines the focus position Z of the objective lens 21 where the sample 12 is in focus. f Identify the position and set the objective lens 21 to the focusing position Z f When installed, the focusing light (reflected light) 228 irradiated onto the sample 12 via the objective lens 21 is received via the objective lens 21, and the received light pattern is acquired as a reference received light pattern, which is stored in the memory unit 44. At this time, the information processing system 100 acquires the reference received light pattern at the focusing position Z where the reference received light pattern was acquired. f The sample 12 is imaged using a microscope 2 equipped with an objective lens 21.

[0053] Furthermore, when the sample 12 is imaged again, the information processing system 100 acquires each light reception pattern when the sample 12 is illuminated with focusing light at each position where the objective lens 21 is driven in the optical axis direction, and uses the similarity between each light reception pattern and the reference light reception pattern stored in the memory unit 44 to determine the imaging reference position (focus position) Z of the objective lens 21 that images the sample 12. f1 Identify the imaging reference position Z f1 Using microscope 2, which has its objective lens set to [specific position], imaging of sample 12 is performed.

[0054] (1-3-3. Effects) The information processing system 100 enables focus detection based on light reflection at the bottom of the sample container, even in cell containers with special shapes that exhibit irregular bottom reflections for cell aggregate formation. In other words, the information processing system 100 can determine the focus position of the objective lens 21 regardless of whether the bottom surface of the sample container is flat or curved, thereby enabling high-speed autofocusing of the microscope 2.

[0055] Furthermore, since the information processing system 100 does not require a long focus search time for imaging exposure time, data transfer time from the imaging device to the processing device, and image analysis time, there is less risk of causing fluorescent dye fading or photodamage to cells when the cell imaging method is epifluorescence or fluorescence confocal. In other words, the information processing system 100 can determine the focus position and perform imaging regardless of the type of sample being imaged.

[0056] [2. Configuration of each device in the information processing system 100] Using Figure 5, the functional configuration of each device in the information processing system 100 shown in Figure 1 will be explained. Below, the configuration example of the information processing device 4 according to the embodiment, the configuration example of the microscope 2, and the processing of the camera 3 will be explained in detail in that order.

[0057] (2-1. Example of Information Processing Device 4 Configuration) First, an example of the configuration of the information processing device 4 shown in Figure 1 will be explained using Figure 5. Figure 5 is a block diagram showing an example of the configuration of each device in the information processing system 100 according to the embodiment. The information processing device 4 has an input unit 41, an output unit 42, a communication unit 43, a storage unit 44, and a control unit 45.

[0058] (2-1-1. Input section 41) The input unit 41 is responsible for inputting various types of information into the information processing device 4. For example, the input unit 41 may be implemented using a mouse or keyboard, and it accepts input such as setting information for the information processing device 4.

[0059] (2-1-2. Output section 42) The output unit 42 is responsible for outputting various types of information from the information processing device 4. For example, the output unit 42 is implemented as a display or the like and outputs setting information stored in the information processing device 4.

[0060] (2-1-3. Communications Section 43) The communication unit 43 is responsible for data communication with other devices. For example, the communication unit 43 communicates data with each communication device via a router or the like. The communication unit 43 can also communicate data with an operator's terminal (not shown).

[0061] (2-1-4. Storage section 44) The storage unit 44 stores various information that the control unit 45 refers to when it operates, and various information acquired when the control unit 45 operates. The storage unit 44 has a light-receiving pattern storage unit 441. Here, the storage unit 44 can be implemented as, for example, a semiconductor memory element such as RAM (Random Access Memory) or flash memory, or a storage device such as a hard disk or optical disc. In the example in Figure 5, the storage unit 44 is installed inside the information processing device 4, but it may be installed outside the information processing device 4, or multiple storage units may be installed.

[0062] (2-1-4-1. Light receiving pattern storage unit 441) The light-receiving pattern storage unit 441 stores the reference light-receiving patterns acquired by the second acquisition unit 453 of the control unit 45. For example, the light-receiving pattern storage unit 441 stores a reference light-receiving pattern for each combination of the type of container 1 and the type of objective lens 21.

[0063] (2-1-5. Control Unit 45) The control unit 45 is responsible for controlling the entire information processing device 4. The control unit 45 includes a first acquisition unit 451, a first identification unit 452, a second acquisition unit 453, a third acquisition unit 454, a second identification unit 455, an imaging unit 456, and a drive unit 457. Here, the control unit 45 can be implemented, for example, by electronic circuits such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or by integrated circuits such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0064] (2-1-5-1. 1st acquisition part 451) The first acquisition unit 451 acquires image data of the sample 12 focused by the objective lens 21 at each position where the objective lens 21 of the microscope 2 is driven in the direction of the optical axis when the object (sample 12) to be observed using the microscope 2 is illuminated. At this time, the first acquisition unit 451 acquires image data captured by the camera 3.

[0065] Furthermore, the first acquisition unit 451 acquires image data for each of the first compartments 11-1 of the container 1, which has multiple compartments 11 and stores a sample 12 in each of the multiple compartments 11. At this time, the first acquisition unit 451 may also acquire image data for a control sample different from the sample 12 stored in the first compartment 11-1 of the container 1. The first acquisition unit 451 may also store the acquired image data in the storage unit 44.

[0066] (2-1-5-2. 1st Specific Part 452) The first identification unit 452 determines the focus position Z of the objective lens 21 where the sample 12 is in focus, based on each image data. f The first identification unit 452 uses the feature quantities of each image data to determine the focus position Z. f The first identification unit 452 identifies the image data that is in focus among the image data using brightness values ​​and contrast values ​​as feature quantities for each image data, and sets the position of the objective lens 21 of the image data to the focus position Z. fIt is identified as such. However, the identification means of the first identification unit 452 is not limited to the means described above.

[0067] (2-1-5-3.Second acquisition part 453) The second acquisition unit 453 moves the objective lens 21 to the focus position Z f When installed, the focused light (reflected light) 228 irradiated onto the sample 12 via the objective lens 21 is received via the objective lens 21, and the received light pattern is acquired as a reference received light pattern, which is stored in the storage unit 44. For example, the second acquisition unit 453 determines that the objective lens 21 identified from the feature quantities of each image data of the first section 11-1 of the container 1 is in focus position Z f The system acquires a reference light reception pattern when installed and stores the reference light reception pattern in the light reception pattern storage unit 441 of the storage unit 44.

[0068] Furthermore, the second acquisition unit 453 stores a reference light-receiving pattern in the light-receiving pattern storage unit 441 of the storage unit 44 for each combination of container type 1 and objective lens type 21. For example, the second acquisition unit 453 stores a reference light-receiving pattern in the light-receiving pattern storage unit 441 of the storage unit 44, such as "Pattern AA" for the combination of container type 1 "Container A" and objective lens type 21 "Objective Lens A", "Pattern AB" for the combination of container type 1 "Container A" and objective lens type 21 "Objective Lens B", and so on.

[0069] Furthermore, the second acquisition unit 453 may store a reference light receiving pattern for each type of container 1 in the light receiving pattern storage unit 441 of the storage unit 44, or it may store a reference light receiving pattern for each type of objective lens 21 in the light receiving pattern storage unit 441 of the storage unit 44. For example, the second acquisition unit 453 stores reference light receiving patterns such as "Pattern A-1", "Pattern B-1", "Pattern C-1", etc. for each type of container 1 "Container A", "Container B", "Container C", etc. in the light receiving pattern storage unit 441 of the storage unit 44. Also, the second acquisition unit 453 stores reference light receiving patterns such as "Pattern A-2", "Pattern B-2", "Pattern C-2", etc. for each type of objective lens 21 "Objective lens A", "Objective lens B", "Objective lens C", etc. in the light receiving pattern storage unit 441 of the storage unit 44.

[0070] (2-1-5-4. Third acquisition part 454) The third acquisition unit 454 acquires each light-receiving pattern when the sample 12 is illuminated with focusing light at each position where the objective lens 21 is driven in the optical axis direction, when the sample 12 is imaged again. For example, when the sample 12 is imaged again, the third acquisition unit 454 can acquire each light-receiving pattern for part or all of the sample 12 being imaged, provided that the container 1 is of the same type and the objective lens 21 is of the same type. The third acquisition unit 454 also acquires each light-receiving pattern for the second section 11-2 of the container 1. That is, when the sample 12 is imaged again, the third acquisition unit 454 can acquire each light-receiving pattern for part or all of the sample 12 being imaged, provided that a reference light-receiving pattern is acquired in the first section 11-1 of the container 1, and then the sample 12 is imaged from the second section 11-2 onwards.

[0071] (2-1-5-5.Second Specific Part 455) The second identification unit 455 uses the similarity between the reference light-receiving pattern stored in the memory unit 44 and each light-receiving pattern to determine the imaging reference position Z of the objective lens 21 that images the sample 12. f1The second identification unit 455 identifies the imaging reference position Z using the similarity between a previously acquired reference light-receiving pattern and each light-receiving pattern acquired during imaging. f1 The second identification unit 455 uses the similarity between the reference light-receiving pattern acquired in the first section 11-1 and each light-receiving pattern acquired in the second section 11-2 to determine the imaging reference position Z f1 The second identification unit 455 identifies the imaging reference position Z in a container 1 having multiple compartments, each compartment containing a sample 12, using the similarity between the reference light-receiving pattern acquired in the first compartment 11-1 for identifying the reference light-receiving pattern and each light-receiving pattern acquired in the second compartment 11-2 for imaging the sample 12. f1 Identify.

[0072] At this time, for example, the second identification unit 455 calculates the cross-correlation value between the reference light-receiving pattern and each light-receiving pattern, and sets the position of the objective lens 21 from which the light-receiving pattern corresponding to the highest cross-correlation value among the cross-correlation values ​​was acquired to the imaging reference position Z f1 Specifically, the second identification unit 455 calculates that the cross-correlation value between the reference light-receiving pattern and "light-receiving pattern A" is "0.7", the cross-correlation value between the reference light-receiving pattern and "light-receiving pattern B" is "0.8", and the cross-correlation value between the reference light-receiving pattern and "light-receiving pattern C" is "0.9", and then identifies the position of the objective lens 21 where "light-receiving pattern C", which has a cross-correlation value, i.e., a similarity close to 1, is obtained, as the imaging reference position Z f1 It is identified as such. The means by which the second identification unit 455 calculates the cross-correlation value are not particularly limited.

[0073] Furthermore, the second identification unit 455 inputs each of the two light-receiving patterns and a reference light-receiving pattern to a machine learning model trained to output the similarity of the two light-receiving patterns in response to the input of two light-receiving patterns, obtains the similarity corresponding to each light-receiving pattern, and sets the position of the objective lens 21 where the light-receiving pattern corresponding to the highest similarity among the similarities was obtained as the imaging reference position Z f1Specifically, the second identification unit 455 identifies the position of the objective lens 21 where "light-receiving pattern C," which has a similarity close to 1, was obtained, if the machine learning model outputs that the similarity between the reference light-receiving pattern and "light-receiving pattern A" is "0.7," the similarity between the reference light-receiving pattern and "light-receiving pattern B" is "0.8," and the similarity between the reference light-receiving pattern and "light-receiving pattern C" is "0.9." f1 It is identified as such. The means by which the second identification unit 455 calculates the similarity are not particularly limited.

[0074] (2-1-5-6. Imaging section 456) The imaging unit 456 acquires the reference light reception pattern at the focus position Z f The microscope 2, in which the objective lens 21 is installed, is used to image the sample 12. For example, when imaging samples 12 of the same type as the container 1 and the same type as the objective lens 21, the imaging unit 456 can transmit a signal including an imaging command to the camera 3, which is an imaging device, and perform imaging of the sample 12. The imaging unit 456 also sets the imaging reference position Z f1 Using a microscope 2 with an objective lens 21 set to the second compartment 11-2, imaging of the sample 12 stored in the second compartment 11-2 is performed. That is, after acquiring a reference light receiving pattern in the first compartment 11-1 of the container 1, the imaging unit 456 transmits a signal including an imaging command to the camera 3, which is an imaging device, when imaging the sample 12 in the second compartment 11-2 and beyond, and can perform imaging of the sample 12 stored in each compartment, such as the second compartment 11-2, the third compartment 11-3, and so on.

[0075] (2-1-5-7. Drive Unit 457) The drive unit 457 controls the drive of the drive device 23 of the microscope 2. For example, the drive unit 457 drives the objective lens 21 connected to the drive device 23 by transmitting a signal to the drive device 23. Specifically, the drive unit 457 drives the objective lens 21 connected to the drive device 23 in the optical axis direction by ΔZ by transmitting a signal to the drive device 23 in order to acquire a light reception pattern. The drive unit 457 also controls the imaging reference position Z f1In order to take an image, the objective lens 21 connected to the drive unit 23 is moved to the imaging reference position Z by transmitting a signal to the drive unit 23. f1 Drive it.

[0076] (2-2. Example of Microscope 2 Configuration) Using Figure 5, an example of the configuration of the microscope 2 shown in Figure 1 will be explained. The microscope 2 has an objective lens 21, a focusing device 22, a drive device 23, and an imaging lens 24.

[0077] (2-2-1. Objective lens 21) The objective lens 21 is installed between the container 1 and the focusing device 22 in the optical axis direction, is driven by the drive device 23, and transmits or focuses illumination light emitted from the illumination device (not shown) during imaging, focusing light emitted from the light-emitting device 221 of the focusing device 22, and focusing light reflected by the bottom surface of the container 1 or the sample 12.

[0078] (2-2-2. Focusing device 22) The focusing device 22 is installed between the objective lens 21 and the imaging lens 24 in the optical axis direction and includes a light-emitting device 221, a first lens 222, a first spectroscopic element 223, a second spectroscopic element 224, a second lens 225, and a two-dimensional optical sensor 226.

[0079] (2-2-2-1. Light-emitting device 221) The light-emitting device 221 irradiates the sample 12 in container 1 with a detection light via the objective lens 21. At this time, the light-emitting device 221 irradiates the sample 12 in container 1 with a detection light based on a signal including a light emission command transmitted by the information processing device 4. The light-emitting device 221 also stops irradiating the detection light based on a signal including a light emission stop command transmitted from the information processing device 4.

[0080] (2-2-2-2. First lens 222) The first lens 222 converts the focusing light (irradiation light) 227 emitted from the light-emitting device 221 into parallel light and guides it to the first spectroscopic element 223.

[0081] (2-2-2-3. First Spectroscopic Element 223) The first spectroscopic element 223 reflects the parallel light of the focusing light (irradiation light) 227 converted by the first lens 222 and guides it to the second spectroscopic element 224. The first spectroscopic element 223 also transmits the focusing light (reflected light) 228 reflected by the second spectroscopic element 224 and guides it to the second lens 225.

[0082] (2-2-2-4. Second Spectroscopic Element 224) The second spectroscopic element 224 reflects the focusing light (irradiation light) 227 reflected by the first spectroscopic element 223 and guides it to the objective lens 21. The second spectroscopic element 224 also reflects the focusing light (reflected light) 228 transmitted by the objective lens 21 and guides it to the second lens 225.

[0083] (2-2-2-5. Second lens 225) The second lens 225 collects the focusing light (reflected light) 228 reflected by the second spectroscopic element 224 and guides it to the two-dimensional light sensor 226.

[0084] (2-2-2-6. 2D optical sensor 226) The two-dimensional optical sensor 226 receives the focused light (reflected light) 228 that has been focused by the second lens 225. The two-dimensional optical sensor 226 also outputs image data of the received light pattern to the information processing device 4.

[0085] (2-2-3. Drive unit 23) The drive unit 23 drives the connected objective lens 21. For example, the drive unit 23 drives the connected objective lens 21 by receiving a signal transmitted by the drive unit 457 of the information processing device 4.

[0086] (2-2-4. Imaging lens 24) The imaging lens 24 is positioned between the focusing device 22 and the camera 3 in the optical axis direction and transmits or focuses the observation light 5 emitted from the sample 12.

[0087] (2-3. Processing of Camera 3) Using Figure 5, we will explain the camera 3, which is the imaging device shown in Figure 1. Camera 3 captures an image of the sample 12 focused by the objective lens 21 at various positions in the optical axis direction when the object (sample 12) to be observed using the microscope 2 is illuminated. For example, camera 3 captures an image of the sample 12 in container 1 by receiving a signal transmitted from the imaging unit 456 of the information processing device 4.

[0088] Furthermore, camera 3 can also capture the light reception pattern of the focusing light (reflected light) 228. For example, by receiving a signal transmitted from the imaging unit 456 of the information processing device 4, it can capture the light reception pattern received by the two-dimensional optical sensor 226.

[0089] Furthermore, the process of imaging the sample 12 in container 1 with camera 3 and acquiring the light reception pattern with the 2D optical sensor 226 has the following advantages. Firstly, since the detection light generally uses a different wavelength than the light used to image the cell sample, if the detection light is to be received by camera 3, the microscope imaging optical system up to camera 3 is restricted, but the above process does not have this disadvantage.

[0090] Secondly, ideally, the focus detection process during cell sample imaging should dynamically match the light-receiving pattern while continuously moving the objective lens 21. Therefore, it is also possible to connect a logic circuit for image processing after the 2D optical sensor 226 to perform a configuration and processing that enables high-speed pattern matching. In this case, the camera 3 for cell sample imaging is often a ready-made camera for embedded use due to the need for high-sensitivity imaging, and adding the above-mentioned circuit is difficult. Therefore, it is preferable to use a light-receiving sensor other than the cell sample imaging camera for the focus detection light.

[0091] [3. Processing flow of information processing system 100] The processing flow of the information processing system 100 according to the embodiment will be explained using Figures 6 and 7. Below, the flow of the reference light reception pattern identification process and the sample imaging process will be explained.

[0092] (3-1. Flow of the process for identifying the reference light reception pattern) The flow of the reference light reception pattern identification process according to the embodiment will be explained using Figure 6. Figure 6 is a flowchart showing an example of the flow of the reference light reception pattern identification process according to the embodiment. Note that the processes in steps S101 to S110 below can be executed in a different order. Also, some of the processes in steps S101 to S110 below may be omitted.

[0093] (3-1-1. Processing of step S101) Firstly, the drive unit 457 of the information processing device 4 controls the drive of the drive device 23 of the microscope 2, moving the objective lens 21 to a predetermined position Z0 (step S101).

[0094] (3-1-2. Processing in step S102) Secondly, the illumination device starts irradiating the sample 12 with light (step S102). At this time, the illumination device may start irradiating based on a signal including an irradiation start command transmitted from the control unit 45 of the information processing device 4.

[0095] (3-1-3. Processing of step S103) Thirdly, camera 3 captures and records an image of sample 12 (step S103). At this time, camera 3 may capture an image of sample 12 based on a signal including an imaging command transmitted from imaging unit 456 of information processing device 4.

[0096] (3-1-4. Processing of step S104) Fourth, the imaging unit 456 of the information processing device 4 checks whether the camera 3 has completed a predetermined number of recordings (step S104). At this time, if the imaging unit 456 has completed the predetermined number of recordings (step S104: Yes), it proceeds to the process in step S106. On the other hand, if the imaging unit 456 has not completed the predetermined number of recordings (step S104: No), it proceeds to the process in step S105.

[0097] (3-1-5. Processing of step S105) Fifth, the drive unit 457 of the information processing device 4 controls the drive of the drive device 23 of the microscope 2, moving the objective lens 21 by ΔZ (step S105), and then returns to the process in step S104.

[0098] (3-1-6. Processing in step S106) Sixth, the illumination device stops irradiating the sample 12 (step S106). At this time, the illumination device may stop irradiating based on a signal including an irradiation stop command transmitted from the control unit 45 of the information processing device 4.

[0099] (3-1-7. Processing of step S107) Seventh, the first identification unit 452 of the information processing device 4 determines the focus position Z based on the image of the sample 12 captured by the camera 3. f Determine (step S107).

[0100] (3-1-8. Processing of step S108) Eighth, the drive unit 457 of the information processing device 4 controls the drive of the drive device 23 of the microscope 2, and moves the objective lens 21 to the focus position Z f Move to (step S108).

[0101] (3-1-9. Processing in step S109) Ninth, the light-emitting device 221 of the microscope 2 starts emitting the focusing light (irradiation light) 227 (step S109). At this time, the light-emitting device 221 may start emitting the focusing light (irradiation light) 227 based on a signal including an emission start command transmitted from the control unit 45 of the information processing device 4.

[0102] (3-1-10. Processing of step S110) Tenth, the two-dimensional optical sensor 226 of the microscope 2 receives the focusing light (reflected light) 228 (step S110). At this time, the second acquisition unit 453 of the information processing device 4 acquires the image data received by the two-dimensional optical sensor 226 as a reference light reception pattern, records it, and terminates processing.

[0103] (3-2. Flowchart of Sample Imaging Process) The flow of the sample imaging process according to the embodiment will be explained using Figure 7. Figure 7 is a flowchart of an example of the sample imaging process according to the embodiment. Note that the processes in steps S201 to S208 below can be executed in a different order. Also, some of the processes in steps S201 to S208 below may be omitted.

[0104] (3-2-1. Processing of step S201) Firstly, the drive unit 457 of the information processing device 4 controls the drive of the drive device 23 of the microscope 2, moving the objective lens 21 to a predetermined position Z0 (step S201).

[0105] (3-2-2. Processing of step S102) Secondly, the light-emitting device 221 of the microscope 2 starts emitting the focusing light (irradiation light) 227 (step S202). At this time, the light-emitting device 221 may start emitting the focusing light (irradiation light) 227 based on a signal including an emission start command transmitted from the control unit 45 of the information processing device 4.

[0106] (3-2-3. Processing of step S203) Thirdly, the two-dimensional optical sensor 226 of the microscope 2 receives the focusing light (reflected light) 228 (step S203). At this time, the third acquisition unit 454 of the information processing device 4 acquires and records the image data received by the two-dimensional optical sensor 226 as a light reception pattern.

[0107] (3-2-4. Processing of step S204) Fourth, the third acquisition unit 454 of the information processing device 4 checks whether the predetermined number of recordings has been completed (step S204). If the third acquisition unit 454 has completed the predetermined number of recordings (step S204: Yes), it proceeds to the process in step S206. On the other hand, if the third acquisition unit 454 has not completed the predetermined number of recordings (step S204: No), it proceeds to the process in step S205.

[0108] (3-2-5. Processing of step S205) Fifth, the drive unit 457 of the information processing device 4 controls the drive of the drive device 23 of the microscope 2, moving the objective lens 21 by ΔZ (step S205), and then returns to the process of step S203.

[0109] (3-2-6. Processing of step S206) Sixth, the light-emitting device 221 of the microscope 2 stops emitting the detection light (irradiation light) 227 (step S206). At this time, the light-emitting device 221 may stop emitting the detection light (irradiation light) 227 based on a signal including an emission stop command transmitted from the control unit 45 of the information processing device 4.

[0110] (3-2-7. Processing of step S207) Seventh, the second identification unit 455 of the information processing device 4 determines the position of the objective lens 21 where the light-receiving pattern of the closest detected light (reflected light) 228 to the reference light-receiving pattern is the imaging reference position Z f1 This is defined (step S207).

[0111] (3-2-8. Processing of step S208) Eighth, camera 3 is positioned at the imaging reference position Z f1 Based on this, an image of sample 12 is captured and recorded (step S208), and the process is terminated. At this time, camera 3 may capture an image of sample 12 based on a signal including an imaging command transmitted from imaging unit 456 of information processing device 4.

[0112] [4. Effects of the Embodiment] Finally, the effects of the embodiment will be described. Below, effects 1 to 7 corresponding to the processing according to the embodiment will be described.

[0113] (4-1. Effect 1) Firstly, in the process according to the above embodiment, when the sample 12 to be observed using the microscope 2 is illuminated, the objective lens 21 of the microscope 2 is driven in the direction of the optical axis, and at each position, image data is acquired of the image of the sample 12 focused by the objective lens 21. Based on each image data, the focus position Z of the objective lens 21 where the sample 12 is in focus is determined. f Identify the position and set the objective lens 21 to the focusing position Z f When installed, the reflected light 228 of the focusing light irradiated onto the sample 12 via the objective lens 21 is received via the objective lens 21, and the received light pattern is acquired as a reference received light pattern, which is then stored in the storage unit 44. Therefore, in the process according to this embodiment, autofocus can be performed at high speed.

[0114] (4-2. Effect 2) Secondly, in the processing according to the embodiment described above, the focus position Z from which the reference light reception pattern was acquired f The sample 12 is imaged using a microscope 2 equipped with an objective lens 21. Therefore, in the process according to this embodiment, autofocus can be performed at high speed, and the sample 12 can be imaged at the same time.

[0115] (4-3. Effect 3) Thirdly, in the process according to the embodiment described above, when the sample 12 is imaged again, each light reception pattern is acquired when the sample 12 is illuminated with focusing light at each position where the objective lens 21 is driven in the optical axis direction, and the similarity between each light reception pattern and the reference light reception pattern stored in the memory unit 44 is used to determine the imaging reference position Z of the objective lens 21 that images the sample 12. f1 Identify the imaging reference position Z f1 The sample 12 is imaged using a microscope 2 equipped with an objective lens 21. Therefore, in the process according to this embodiment, autofocus can be performed at high speed and the image of the sample 12 can be effectively performed.

[0116] (4-4. Effect 4) Fourth, in the processing according to the embodiment described above, image data is acquired for each of the first compartments 11-1 of a container 1 having a plurality of compartments 11, each of which stores a sample 12, and the focus position Z is determined using the feature quantities of each image data. f The objective lens 21 is identified and focuses at position Z f When installed, a reference light receiving pattern is acquired, the reference light receiving pattern is stored in the storage unit 44, each light receiving pattern is acquired for the second section 11-2 of the container 1, and the similarity between the reference light receiving pattern and each light receiving pattern is used to determine the imaging reference position Z f1 Identify the imaging reference position Z f1 Using a microscope 2 equipped with an objective lens 21, imaging of the sample 12 stored in the second compartment 11-2 is performed. Therefore, in the process according to this embodiment, high-speed autofocus can be performed on the container 1 having multiple compartments 11, and imaging of the sample 12 can be performed effectively.

[0117] (4-5. Effect 5) Fifth, in the processing according to the embodiment described above, a reference light-receiving pattern is stored in the storage unit 44 for each combination of container type 1 and objective lens type 21. Therefore, in the processing according to the embodiment, when the container type 1 and objective lens type 21 are the same, autofocus can be performed at a faster speed and imaging of the sample 12 can be performed effectively.

[0118] (4-6. Effect 6) Sixth, in the processing according to the embodiment described above, the cross-correlation value between the reference light-receiving pattern and each light-receiving pattern is calculated, and the position of the objective lens 21 that obtained the light-receiving pattern corresponding to the highest cross-correlation value among the cross-correlation values ​​is set to the imaging reference position Z f1 This is identified. Therefore, in the process according to the embodiment, autofocus can be performed more effectively and quickly using the cross-correlation value, and imaging of the sample 12 can be performed effectively.

[0119] (4-7. Effect 7) Seventh, in the processing according to the embodiment described above, each of the two light-receiving patterns and the reference light-receiving pattern are input to a machine learning model trained to output the similarity of two light-receiving patterns in response to the input of two light-receiving patterns, each similarity corresponding to each light-receiving pattern is obtained, and the position of the objective lens 21 from which the light-receiving pattern corresponding to the highest similarity among the similarities was obtained is set to the imaging reference position Z f1 This is identified. Therefore, in the process according to the embodiment, autofocus can be performed more effectively and quickly using a machine learning model, and imaging of the sample 12 can be performed effectively.

[0120] 〔system〕 Unless otherwise specified, the processing procedures, control procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed at will.

[0121] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown. That is, all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0122] Furthermore, each processing function performed by each device may be implemented, in whole or in part, by a CPU and a program executed for analysis by that CPU, or by wired logic hardware.

[0123] [Hardware] Next, an example of the hardware configuration of the information processing device 4 will be described. Figure 8 is a diagram illustrating an example of the hardware configuration. As shown in Figure 8, the information processing device 4 includes a communication device 4a, an HDD (Hard Disk Drive) 4b, memory 4c, and a processor 4d. Furthermore, each of the parts shown in Figure 8 is interconnected by a bus or the like.

[0124] The communication device 4a is a network interface card or similar, and is used for communication with other servers. The HDD 4b stores the programs and databases that operate the functions shown in Figure 5.

[0125] The processor 4d operates a process that performs the functions described in Figure 5 by reading a program from the HDD 4b or the like that performs the same processing as each processing unit shown in Figure 5 and loading it into memory 4c. For example, this process performs the same functions as each processing unit of the information processing device 4. Specifically, the processor 4d reads a program from the HDD 4b or the like that has the same functions as the first acquisition unit 451, first identification unit 452, second acquisition unit 453, third acquisition unit 454, second identification unit 455, imaging unit 456, drive unit 457, etc. Then, the processor 4d executes a process that performs the same processing as the first acquisition unit 451, first identification unit 452, second acquisition unit 453, third acquisition unit 454, second identification unit 455, imaging unit 456, drive unit 457, etc.

[0126] Thus, the information processing device 4 operates as a device that executes various processing methods by reading and executing a program. Furthermore, the information processing device 4 can also achieve the same functionality as the embodiment described above by reading the program from the recording medium using a media reading device and executing the read program. It should be noted that the program referred to in this other embodiment is not limited to being executed by the information processing device 4. For example, the present invention can be similarly applied when another computer or server executes a program, or when they collaborate to execute a program.

[0127] This program can be distributed via networks such as the Internet. Furthermore, this program can be recorded on computer-readable storage media such as hard disks, flexible disks (FDs), CD-ROMs, MO (Magneto-Optical disks), and DVDs (Digital Versatile Discs), and executed by reading the program from these media using a computer. [Explanation of Symbols]

[0128] 1 container 11 Compartments of the container 2 Microscopes 21 Objective lens 22 Focusing device 221 Light-emitting device 222 First Lens 223 First Spectroscopic Element 224 Second Spectroscopic Element 225 Second lens 226 2D optical sensor 227 Focusing light (irradiation light) 228 Focusing light (reflected light) 3 cameras 4. Information Processing Device 41 Input section 42 Output section 43 Communications Department 44 Storage section 441 Light receiving pattern storage unit 45 Control Unit 451 First acquisition part 452 1st Specific Part 453 Second Acquisition Department 454 Third Acquisition Department 455 Second Specific Part 456 Imaging Unit 457 Drive Unit 5 Observation light 100 Information Processing Systems

Claims

1. A first acquisition unit acquires first image data, each image of the object focused by the objective lens, at each position where the objective lens of the microscope is driven in the direction of the optical axis when illumination is shone on the object to be observed using the microscope. A first identification unit that identifies the focus position of the objective lens where the object is in focus based on each of the first image data, When the objective lens is placed at the specified focusing position, the second acquisition unit acquires a light-receiving pattern, which is a second image data received through the objective lens, from the reflected light of the focusing light irradiated onto the object through the objective lens, and stores the reference light-receiving pattern in the storage unit. When the object is imaged again, a third acquisition unit acquires each light reception pattern when the object is illuminated with the focusing light at each position where the objective lens is driven in the optical axis direction, A second identification unit identifies the imaging reference position of the objective lens that images the object, using the similarity between the reference light-receiving pattern stored in the memory unit and each of the light-receiving patterns, An information processing device equipped with the following features.

2. An imaging unit performs imaging of the object using the microscope in which the objective lens is installed at the focal position where the reference light receiving pattern is acquired. The information processing apparatus according to claim 1, further comprising:

3. The imaging unit is Using the microscope in which the objective lens is set at the aforementioned imaging reference position, imaging of the object is performed. The information processing apparatus according to claim 2.

4. The first acquisition unit is, For a container having multiple compartments, each of which stores an object, the first image data is acquired for each of the first compartments. The first specified part is, Using the feature quantities of each of the first image data described above, the focus position is identified, The aforementioned second acquisition unit is, The reference light receiving pattern is obtained when the objective lens is installed at the focusing position, and the reference light receiving pattern is stored in the storage unit. The aforementioned acquisition unit is: For the second compartment of the container, each of the light receiving patterns is acquired. The second specified part is, The imaging reference position is identified using the similarity between the reference light-receiving pattern and each of the light-receiving patterns. The imaging unit is Using the microscope in which the objective lens is installed at the aforementioned imaging reference position, imaging of the object stored in the second compartment is performed. The information processing apparatus according to claim 3.

5. The aforementioned second acquisition unit is, For each combination of the container type and the objective lens type, the reference light receiving pattern is stored in the storage unit. The information processing apparatus according to claim 4.

6. The second specified part is, The cross-correlation value between the reference light-receiving pattern and each of the light-receiving patterns is calculated, and the position of the objective lens where the light-receiving pattern corresponding to the highest cross-correlation value is acquired is identified as the imaging reference position. The information processing apparatus according to claim 4.

7. The second specified part is, A machine learning model, trained to output the similarity between two light-receiving patterns in response to the input of two light-receiving patterns, is input with each of the light-receiving patterns and the reference light-receiving pattern. The similarity corresponding to each of the light-receiving patterns is obtained, and the position of the objective lens from which the light-receiving pattern with the highest similarity is obtained is identified as the imaging reference position. The information processing apparatus according to claim 4.

8. In an information processing system comprising an imaging device, a microscope, and an information processing device, The imaging device is When the object to be observed using the microscope is illuminated, the objective lens of the microscope is driven in the direction of the optical axis, and at each position, an image of the object focused by the objective lens is captured. The aforementioned microscope, The device includes a light-emitting device that illuminates the object with a focusing light through the objective lens, The aforementioned information processing device is A first acquisition unit acquires each first image data of the object captured by the imaging device, A first identification unit that identifies the focus position of the objective lens where the object is in focus based on each of the first image data, When the objective lens is placed at the specified focusing position, the second acquisition unit acquires a light-receiving pattern, which is a second image data obtained by receiving the reflected light of the focusing light irradiated onto the object by the light-emitting device through the objective lens, as a reference light-receiving pattern, and stores the reference light-receiving pattern in the storage unit. When the object is imaged again, a third acquisition unit acquires each light reception pattern when the object is illuminated with the focusing light at each position where the objective lens is driven in the optical axis direction, A second identification unit identifies the imaging reference position of the objective lens that images the object, using the similarity between the reference light-receiving pattern stored in the memory unit and each of the light-receiving patterns, An information processing system having

9. Computers When an object to be observed using a microscope is illuminated, the objective lens of the microscope is driven in the direction of the optical axis, and at each position, a first image data is acquired in which an image of the object focused by the objective lens is captured. Based on each of the first image data described above, the focusing position of the objective lens where the object is in focus is identified. When the objective lens is placed at the specified focusing position, the reflected light of the focusing light irradiated onto the object through the objective lens is received through the objective lens, and the received light pattern, which is a second image data, is acquired as a reference received light pattern, and the reference received light pattern is stored in the storage unit. When imaging the object again, each light reception pattern is acquired when the object is illuminated with the focusing light at each position where the objective lens is driven in the optical axis direction. The imaging reference position of the objective lens that images the object is determined using the similarity between the reference light-receiving pattern stored in the memory unit and each of the light-receiving patterns. An information processing method that performs a process.

10. On the computer, When an object to be observed using a microscope is illuminated, the objective lens of the microscope is driven in the direction of the optical axis, and at each position, a first image data is acquired in which an image of the object focused by the objective lens is captured. Based on each of the first image data described above, the focusing position of the objective lens where the object is in focus is identified. When the objective lens is placed at the specified focusing position, the reflected light of the focusing light irradiated onto the object through the objective lens is received through the objective lens, and the received light pattern, which is a second image data, is acquired as a reference received light pattern, and the reference received light pattern is stored in the storage unit. When imaging the object again, each light reception pattern is acquired when the object is illuminated with the focusing light at each position where the objective lens is driven in the optical axis direction. The imaging reference position of the objective lens that images the object is determined using the similarity between the reference light-receiving pattern stored in the memory unit and each of the light-receiving patterns. An information processing program that executes a process.

Images