Focus adjustment method, program, and apparatus

Abstract

The technology of the present disclosure provides a novel focus adjustment method.SOLUTION: A learning method for an estimator that estimates, using a microscope including an objective lens, a moving direction of a focal point of the objective lens with respect to an in-focus position of a first subject, the learning method comprising: capturing, using the microscope including the objective lens, a second subject a plurality of times while changing a position of the objective lens in an optical axis direction with respect to the second subject at predetermined intervals, using a diaphragm having a diameter smaller than a full aperture; A learning method comprising: an acquisition process of acquiring at least two second microscopic images; a training data generation process of creating an image of a set of at least two divided second microscopic images showing the same portion of the second subject acquired in the acquisition process; and a learning process of performing machine learning of the created image of the set of the second microscopic images.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a focus adjustment method, a program, and an apparatus. [Background technology]

[0002] Conventionally, a method for calculating a focal position based on an image acquired by a microscope is known (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2020-060602 Summary of the Invention [Problem to be solved by the invention]

[0004] The techniques of this disclosure provide a novel focus adjustment method. [Means for solving the problem]

[0005] One embodiment of the present invention is a focus adjustment method that includes an acquisition process in which a first object is photographed multiple times using a microscope equipped with an objective lens, with an aperture having a diameter smaller than the maximum aperture, while changing the position of the objective lens in the optical axis direction relative to the first object at predetermined intervals, thereby acquiring two first microscope images; an estimation process in which the two first microscope images are input into a trained model that estimates the movement direction of the focus of the objective lens relative to the focus position of the first object, thereby estimating the movement direction; and a process in which the focus is moved relative to the object based on the movement direction. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is an overall configuration diagram of an information processing system according to an embodiment. [Figure 2] FIG. 1 is a diagram illustrating a hardware configuration of an information processing device. [Figure 3]FIG. 2 is a diagram illustrating a functional configuration of a server. [Figure 4] FIG. 2 is a diagram illustrating a configuration of a learning device. [Figure 5] FIG. 1 is a diagram illustrating the configuration of a dataset. [Figure 6] 1 is a flowchart illustrating a process according to an embodiment. [Figure 7] FIG. 10 is a diagram showing the relationship between an aperture stop and a depth of focus. [Figure 8] FIG. 10 is a diagram showing the relationship between an image captured by a microscope device and divided images. [Figure 9] 10A and 10B are diagrams illustrating images used in the learning process and the estimation process. [Figure 10] 10A to 10C are diagrams illustrating the operation of the objective lens in the estimation process. [Figure 11] 10A and 10B are diagrams illustrating the relationship between images used in the estimation process and estimation results. DETAILED DESCRIPTION OF THE INVENTION

[0007] Hereinafter, the present invention will be described based on one embodiment thereof with reference to the drawings. 〔composition〕 1 shows the configuration of an information processing system 1 according to one embodiment of the present invention. The information processing system 1 includes a server 10, a terminal 20, and a microscope device 30. The server 10, the terminal 20, and the microscope device 30 are connected via a network 5 so as to be able to send and receive data to and from each other.

[0008] The network 5 is a wireless or wired communication means, such as the Internet, a WAN (Wide Area Network), a LAN (Local Area Network), a public communication network, or a dedicated line. Although the information processing system 1 according to this embodiment is configured by a plurality of information management devices, the present invention does not limit the number of these devices. Therefore, the information processing system 1 can be configured by one or more devices as long as they have the following functions:

[0009] The server 10 and the terminal 20 control the microscope device 30, acquire images captured by the microscope device 30, and edit and analyze the images.

[0010] The microscope device 30 is a device that uses a microscope to photograph (also referred to as image capture) biological tissue as a subject. One example of the subject is a cell. As shown in FIG. 1 , the microscope device 30 includes an objective lens 31, a stage 32, an aperture stop 33, a revolver 34 that supports multiple objective lenses 31, a light source 35, a reflecting mirror 36, a condenser lens 37, an eyepiece 45, and an image capture unit 46 (including a CCD sensor, a CMOS sensor, etc.), and can capture images of biological tissue samples placed in each of wells W1 and W2 of a well plate P.

[0011] In addition, the microscope device 30 is equipped with a reflecting mirror 39, positive and negative lenses 40A and 40B, a half mirror 41, a lens 42, and relay optical systems 43A and 43B, and these components form an optical path that guides light from the objective lens 31 to the eyepiece lens 45 and the imaging unit 46.

[0012] The microscope device 30 can change the position in the Z axis direction (referred to as the Z position) of the objective lens 31 relative to the stage 32 by moving the objective lens 31 along the Z axis extending vertically, thereby changing the distance between the focal point F (FIG. 10) of the objective lens 31 and the sample placed on the well plate P on the stage 32. In addition, the microscope device 30 can move the stage 32 along X and Y axes extending horizontally and perpendicular to each other. These X, Y, and Z position signals are output as position signals by encoders or the like in the drive units of the objective lens 31 and stage 32 (not shown), and are stored in a storage device. The Z position can be changed by changing the relative position between the objective lens 31 and the stage 32, so the stage 32 may be moved in the Z-axis direction.

[0013] The aperture stop 33 is disposed on the optical path between the sample and the light source 35. In this embodiment, it is disposed above the condenser lens 37.

[0014] A light beam emitted from a light source 35 (such as a tungsten lamp) is irradiated onto the sample via a reflecting mirror 36, an aperture stop 33, and a condenser lens 37. The light source 35 may also be a point light source.

[0015] Light from the sample passes through the objective lens 31 and lenses 40A and 40B, and is then split into two by a half mirror 41 and directed to a visual optical path and a photographing optical path, respectively.

[0016] The light in the visual optical path is guided to an eyepiece 45 via a reflecting mirror 39 and relay optical systems 43A and 43B. The light in the photographing optical path passes through a lens 42 and is guided to an imaging unit 46, where an image is captured.

[0017] The microscope device 30 according to this embodiment includes an optical microscope and acquires bright-field images. Examples of images acquired by an optical microscope include phase-contrast images, bright-field images, differential interference contrast images, confocal microscope images, super-resolution microscope images, fluorescent images, and stained images used in pathological diagnosis.

[0018] 2 shows an example of hardware (hereinafter referred to as "information processing device 100") used to realize server 10 and terminal 20. As shown in the figure, information processing device 100 includes a processor 101, a main memory device 102, an auxiliary memory device 103, an input device 104, an output device 105, and a communication device 106. These are connected to each other so as to be able to communicate with each other via communication means such as a bus (not shown).

[0019] Note that the entire configuration of server 10 does not necessarily need to be realized by hardware, and all or part of the configuration may be realized by virtual resources such as a cloud server of a cloud system.

[0020] The processor 101 is configured using a CPU (Central Processing Unit), an MPU (Micro Processing Unit), etc. The processor 101 reads and executes a program stored in a main memory device 102, thereby realizing the functions of the server 10 and the terminal 20.

[0021] The main memory device 102 is a device that stores programs and data, and is a ROM (Read Only Memory), a RAM (Random Access Memory), a non-volatile semiconductor memory (NVRAM (Non-Volatile RAM)), etc. The auxiliary memory device 103 is various non-volatile memories (NVRAM: Non-volatile memory) such as an SSD (Solid State Drive) or an SD memory card, a hard disk drive, an optical storage device (CD (Compact Disc), DVD (Digital Versatile Disc), etc.), a storage area of ​​a cloud server, etc.

[0022] The input device 104 is an interface that accepts input of information, and is, for example, a keyboard, a mouse, a touch panel, a card reader, a voice input device (such as a microphone), a voice recognition device, etc. The image processing device 100 may be configured to accept input of information between it and other devices via the communication device 106.

[0023] The output device 105 is an interface that outputs various types of information, and is, for example, a screen display device (such as a liquid crystal monitor, LCD (Liquid Crystal Display), or graphic card), a printer, an audio output device (such as a speaker), or an audio synthesizer. The image processing device 100 may be configured to output information to and from other devices via the communication device 106. The output device 105 corresponds to the display unit of the present invention.

[0024] The communication device 106 is a wired or wireless communication interface that enables communication with other devices via the network 5, such as a NIC (Network Interface Card), a wireless communication module, a USB (Universal Serial Interface) module, a serial communication module, etc.

[0025] [Functional configuration] 3 shows the main functional configuration of the server 10. As shown in the figure, the server 10 includes a database 114 and a management unit 120.

[0026] The database 114 is stored in the main storage device 102 or the auxiliary storage device 103 of the server 10. The database 114 stores data sets D and D1 used for machine learning (FIG. 5).

[0027] The data sets D and D1 hold a plurality of combinations of images R1 to R9 associated with relative positions L1 to L9, each numbered as shown in Fig. 5. The method of generating the data sets D and D1 will be described later.

[0028] As shown in FIG. 8, images R1 to R9 are images obtained by dividing image R, which is acquired by capturing images of biological tissue such as cells using the imaging unit 46, into 3×3 sections. Furthermore, relative positions L1 to L9 indicate the relative position of the focal point F of the objective lens 31 in the Z-axis direction with respect to the focal point FP (FIG. 10) of the sample (image R) captured in images R1 to R9. Here, the focal point FP indicates the Z position at which the focal point F can be positioned to capture an in-focus image of the sample. In other words, the relative positions L1 to L9 indicate how far the sample or the objective lens 31 should be moved, either vertically or horizontally, to capture an in-focus image of the sample. The focal point FP and relative positions L1 to L9 are determined based on the Z position signal of the objective lens 31.

[0029] In addition to the above functions, the server 10 also has functions such as an operating system, a file system, a device driver, and a DBMS (DataBase Management System).

[0030] The management unit 120 performs processes executed by the server 10, such as image acquisition and management. The functions of the management unit 120 are realized by the processor 101 of the server 10 reading and executing a program stored in the main storage device 102 or the auxiliary storage device 103 of the server 10. The management unit 120 includes a learning device 116.

[0031] The learning device 116 can learn the feature quantities of the input image. The learning device 116 constructs a neural network that outputs, for the input image, an estimation result of the relative position of the focal point F of the objective lens 31 with respect to the in-focus position FP. For example, the neural network is a deep convolutional neural network (DCNN).

[0032] The learning device 116 has an input layer that receives an image input, an output layer that outputs an estimation result of an element of interest, and an intermediate layer that extracts features of the input image (FIG. 4). Each of the input layer, output layer, and intermediate layer has nodes (shown by white circles in the figure), and the nodes of each of these layers are connected by edges (shown by arrows in the figure). Note that the configuration of the learning device 116 shown in FIG. 4 is an example, and the number of nodes and edges, the number of intermediate layers, etc. can be changed as appropriate.

[0033] [Processing details] The following describes in detail the processing executed in the information processing system 1. As shown in FIG. 6, the processing of the information processing system 1 includes two types of processing: a learning processing and an estimation processing.

[0034] (Learning process) Learning by the learner 116 is performed based on the flowchart shown in Fig. 6. When the server 10 receives a user instruction via the terminal 20, the processor 101 of the server 10 starts a program stored in the main storage device 102 or the auxiliary storage device 103. Furthermore, the management unit 120 executes the following processing.

[0035] In the following description, the processing executed by the management unit 120 of the server 10 may be simply described as being executed by the "server 10."

[0036] First, a well plate P on which a sample is placed is placed in the microscope device 30 (S1).

[0037] The management unit 120 narrows the aperture diaphragm 33 of the microscope device 30 to a smaller aperture diameter than the maximum aperture (S3). It is particularly desirable to narrow the aperture to the minimum diameter. For bright-field images, instead of adjusting the aperture diaphragm, a small light source such as a point light source may be placed at the position of the aperture diaphragm 33. For images other than bright-field images, such as fluorescence images, the above-pupil diaphragm on the detection lens side may be narrowed. It is desirable for the above-pupil diaphragm to be located in a position conjugate with the aperture diaphragm. Alternatively, a diaphragm located at the exit pupil may be used instead of adjusting the aperture diaphragm. Narrowing the aperture diaphragm 33 extends the focal depth of the objective lens 31, thereby increasing the range of Z positions at which the sample image can be clearly captured (FIG. 7). Furthermore, the amount of change in contrast with respect to a change in the Z position of the focal point F also increases. This makes it easier to grasp the distance between the focal point F and the in-focus position FP of the sample.

[0038] Furthermore, the management unit 120 causes the microscope device 30 to perform Z-stack imaging of the sample (S5). In this process, the microscope device 30 images the sample multiple times while shifting the Z position of the objective lens 31 by ΔZ within a predetermined range, thereby obtaining multiple images R. The range in which the sample is imaged can be set by the user, and as an example, it can be a Z position range of ±300 micrometers (μm) from a reference position. The amount of movement ΔZ of the objective lens 31 is +20 μm in this example, and includes information on the movement direction and amount of movement of the objective lens 31. Note that ΔZ is not limited to +20 μm, and can be set to any value, such as -2 μm or +10 μm.

[0039] Once the photographing is complete, the management unit 120 performs preprocessing on the multiple images R acquired by photographing (S7). In the preprocessing, each image R is divided into nine parts to generate nine images R1 to R9 (FIG. 8). The management unit 120 analyzes the contrast of the images R1 to R9 obtained by photographing the sample, and calculates, for each image R1 to R9, the relative positions L1 to L9 of the focal point F in the Z-axis direction relative to the in-focus position FP of the image R. The in-focus position FP is obtained by calculating the Z position of the focal point F when the image is photographed with the minimum contrast. The relative positions L1 to L9 are expressed as numerical values ​​obtained by adding a ± sign indicating up or down to the distance in the Z direction, as shown in FIGS. 5 and 9.

[0040] The images R1 to R9 and the relative positions L1 to L9 are associated with each other and saved as a data set D (S9, FIG. 5).

[0041] In step S9, the management unit 120 uses the data acquired in step S9 as training data and causes the learning device 116 to learn (S11). More specifically, for each of images R1 to R9, the management unit 120 extracts two images that are adjacent in the shooting order (shown as No. in data set D) and stores the relative positions L1 to L9 corresponding to these two images as labels (FIG. 5(b), data set D1). In other words, for example, the captured images R of No. 1 and No. 2 are paired with each other, with the Z positions at the time of shooting differing by ΔZ, and each image (R1 to R9) showing the same position on the sample. For the label attached to each pair of images R1 to R9, the larger or smaller value of the relative positions associated with the two images is always selected.

[0042] In this embodiment, the larger value is always selected and used as the label. As shown in data sets D and D1 in Fig. 5, the image R1 of the pair of No. 1 and No. 2 is labeled with the value of the relative position L1 of No. 1, and the image R1 of the pair of No. 2 and No. 3 is labeled with the value of the relative position L1 of No. 2.

[0043] The combinations of the two images stored in dataset D1 and the associated labels are input to the learning device 116. As shown in FIG. 9, the management unit 120 extracts two images, No. 1 and No. 2, from each of images R1 to R9, and sequentially inputs them together with their labels to the learning device 116. Next, the management unit 120 extracts two images, No. 2 and No. 3, from each of images R1 to R9, and sequentially inputs them together with their labels to the learning device 116. In this way, the management unit 120 causes the learning device 116 to learn all of the acquired combinations of images and labels.

[0044] As a result of the learning process, the learning device 116 functions as a learned model that estimates the position (movement direction and amount) of the focal point F relative to the focus position FP of the displayed sample for the input image (S13).

[0045] (Estimation process) By using the trained learning device 116 generated by the learning process, it is possible to estimate the position of the focal point F in a short time and quickly align the focal point F with the focusing position FP of the sample. An example of the estimation process using the trained learning device 116 will be described below with reference to the flowchart in FIG.

[0046] First, a well plate P in which samples are placed in wells W1 is installed in the microscope device 30 (S21). Note that the samples used in the estimation process are not limited to the same samples as those used in the learning process.

[0047] The management unit 120 sets the aperture diaphragm 33 of the microscope device 30 to a small aperture diameter (S23). It is particularly desirable to narrow it down to the minimum diameter. Furthermore, for bright-field images, a small light source such as a point light source may be used instead of adjusting the aperture diaphragm. For images other than bright-field images, such as fluorescence images, the diaphragm above the pupil on the detection lens side may be narrowed. By narrowing the aperture diaphragm 33, the depth of focus increases, making it easier to grasp the distance between the focus F and the in-focus position FP.

[0048] In step S23, initialization is performed. The microscope device 30 locates the bottom surface of the well W1 containing the sample by irradiating it with PFS light, and moves the stage 32 so that the focal point F is located on the top surface of the bottom of the well W1 (FIG. 10).

[0049] Next, the management unit 120 causes the microscope device 30 to photograph the sample (S25). After photographing the sample for the first time, the microscope device 30 moves the objective lens 31 by ΔZ and performs a second photographing. The management unit 120 acquires two images Q1 and Q2 obtained by photographing from the microscope device 30.

[0050] When the image capturing is completed, the management unit 120 performs pre-processing on the images Q1 and Q2 (S27). In the pre-processing, the images Q1 and Q2 are each divided into nine parts to generate images Q11 to Q19 and Q21 to Q22 (FIG. 11).

[0051] In step S29, the management unit 120 inputs images Q11 to Q19 and Q21 to Q22 into the learning device 116, and causes it to estimate the position (movement direction and amount) of the focal point F relative to the in-focus position FP. In detail, as shown in Fig. 11, from a total of 18 images Q11 to Q19 and Q21 to Q22, the management unit 120 creates combinations of images Q11 and Q21, a set of images Q12 and Q22, ..., a set of images Q19 and Q29, and combinations of two images showing the same location on the sample, and these are called combinations T1 to T9. The management unit 120 sequentially inputs the combinations T1 to T9 into the learning device 116 (Figs. 9 and 11).

[0052] The learning device 116 estimates and outputs the relative position of the focal point F with respect to the in-focus position FP of the sample for each of the combinations T1 to T9. In this embodiment, for each of the combinations T1 to T9, the estimation result for the image showing the larger estimated value of the two images constituting the combination is output. This is because in the learning process, the relative positions L1 to L9 with the larger value were used as labels for each pair of images R1 to R9.

[0053] The management unit 120 calculates a representative value of the estimation results for the combinations T1 to T9 and outputs it as an estimated position (S31). As an example, the representative value of the estimated position is the median (FIG. 11). Note that the calculation of the representative value is not limited to the median, and other mathematical processes such as the average value can also be used.

[0054] The management unit 120 moves the objective lens 31 of the microscope device 30 based on the estimated position (movement direction and movement amount) obtained as a result of the estimation process to change the position of the focal point F and align the focal point F with the in-focus position FP (S33). Note that the information on the estimated position obtained as a result of the estimation process may be only the movement direction, and the objective lens 31 may be controlled to stop when it reaches the in-focus position.

[0055] If it is considered that the focus F is not on the sample (S33: NO), the management unit 120 returns the process to step S25, further narrows the aperture stop 33, and redoes the processes from S25 onwards.

[0056] If the position of the focal point F is aligned with the in-focus position FP (S35: YES), the management unit 120 changes the sample to be photographed and repeats the estimation process (S39: NO). For example, if the sample used in the estimation process is the sample in well W1, the microscope device 30 moves the stage 32 to change the target of the estimation process to the sample in well W2 (FIG. 10), and repeats the processes from step S25 onwards (S37).

[0057] <Effects> The focus adjustment method of the above embodiment includes an acquisition process (S25) in which the diameter of an aperture (corresponding to the aperture of the present invention) such as an aperture stop 33 is reduced or a point light source is used, and a microscope device 30 equipped with an objective lens 31 is used to photograph a sample (corresponding to the first subject) multiple times while changing the position of the objective lens 31 at regular intervals to acquire a microscope image R (corresponding to the first microscope image set); a process (S27 to S31, corresponding to the estimation process) in which the microscope image R is input to a trained learning device 116 that estimates the position of the focal point F of the objective lens 31 relative to the focus position of the sample, to estimate the position of the focal point F relative to the focus position FP and use it as the estimated position; and a process (S33) in which the focal point F is moved relative to the sample based on the estimated position.

[0058] By performing the above-described processing, the microscope device 30 can quickly focus on the sample and take an in-focus image.

[0059] In the above embodiment, the microscope device 30 executes the acquisition process (S25) and the estimation process (S31) with the diameter of the aperture stop 33 set to the minimum diameter. Also, if the estimation process (S31) is inappropriate and the focus is not achieved, the process is executed again (S33: NO). The aperture stop 33 is preferably located closer to the light source than the sample.

[0060] By setting the aperture diameter to the minimum, the focal depth of the objective lens 31 is extended, and the Z position range of the focus F in which the image of the sample can be clearly captured is increased. In addition, the amount of change in contrast with changes in the Z position of the focus F also increases. This makes it easier to grasp the distance between the focus F and the in-focus position FP of the sample.

[0061] In the above embodiment, the estimation process (S27 to S31) includes a process (S27) of dividing the microscopic images Q1 and Q2 to generate multiple images Q11 to Q19 and Q21 to Q29 (corresponding to partial images), a process (S29) of inputting the images to the learning device 116 and estimating the positions of the focus F relative to the in-focus position FP for the sets T1 to T9 of images Q11 to Q19 and Q21 to Q29, and a process (S31) of setting the representative value of the estimated values ​​as the estimated position.

[0062] By dividing the image as described above and performing estimation for each part, it is possible to obtain an accurate estimated position.

[0063] Furthermore, by using multiple microscope images in the estimation process, the change in the image in response to movement of the focal point F can be grasped, and the learning device 116 can accurately estimate the distance of the focal point F relative to the in-focus position FP. Furthermore, by grasping the change in the image, the learning device 116 can grasp whether the focal point F moved away from the in-focus position FP or whether the focal point F moved closer to the in-focus position FP when the multiple images Q1 and Q2 were captured. This allows the learning device 116 to grasp not only the distance but also whether the focal point F is located above or below the in-focus position FP. Therefore, the learning device 116 can accurately estimate not only the distance but also the relative position of the focal point F relative to the in-focus position FP.

[0064] In the above embodiment, the trained model is generated by taking multiple photographs using an aperture with a diameter smaller than the maximum aperture while changing the position of the objective lens 31 in the optical axis direction relative to the sample at predetermined intervals within a predetermined range, obtaining two images R (second microscope images) (S5), calculating the relative position of the focus of the objective lens 31 relative to the focal position of the sample for each of the two images R (S9), and having the learner 116 train the training data D1 including a combination of the two images R and the associated relative positions (S11, S13).

[0065] The trained model is generated by dividing each of two images R to generate a plurality of partial images R1 to R9, calculating the relative position of the focus of the objective lens 31 relative to the focal position of the sample for each of the plurality of partial images R1 to R9, and having the learner 116 machine-learn the training data D1 including combinations of the plurality of partial images R1 to R9 and the associated relative positions, and in the combinations, the associated relative positions are selected as either the larger value or the smaller value of the relative positions corresponding to the plurality of partial images R1 to R9.

[0066] As described above, by using images obtained by a shooting procedure similar to that of the estimation process as training data for the learner 116, it is possible to generate a trained model that performs estimation processing with high accuracy. Furthermore, by learning combinations of multiple images together with labels, the learner 116 learns, based on changes in the multiple images, whether the focal point F has moved away from the in-focus position FP or whether the focal point F has moved closer to the in-focus position FP. As a result of learning, the learner 116 functions as a trained model that accurately estimates not only the distance but also the relative position of the focal point F with respect to the in-focus position FP.

[0067] <Modification> The division method and number of divisions for image R, image Q1, and image Q2 are arbitrary. Therefore, each image can be divided into any shape and any number of images, not limited to 9 divisions as in the above embodiment, such as 16 divisions or 32 divisions. Also, image R, image Q1, and image Q2 may be input directly to the learning device 116 without division.

[0068] In the above embodiment, the number of images input to the learning device 116 in the estimation process and learning process is not limited to two. For example, three or more input images can be used. In this case, the relative position of the focal point F with respect to the focus position FP in any of the multiple input images can be used as the label associated with each combination. Furthermore, the machine learning method in the above embodiment may be regression or classification. When machine learning by classification is performed, interpolation between data in post-processing can be performed to perform learning similar to that described above and generate an estimation model.

[0069] The well plate P on which the samples are placed does not need to be the same in the estimation process and the learning process. It is also possible to prepare multiple types of containers to place samples in and perform the learning process, thereby generating trained models that can accommodate containers of various shapes and materials.

[0070] In the above embodiment, a plurality of terminals are connected to one server 10 to perform the above-described functions. The present invention does not limit the number of servers or the number of terminals, and for example, the above-described functions may be performed by only one device. The number of terminals or the number of servers may also be increased. Furthermore, each function does not necessarily have to be performed by the server 10 or the like, and the functions may be shared and performed by a plurality of devices. In other words, the present invention does not limit the number of control units or devices, or the sharing of functions among devices. [Explanation of symbols]

[0071] 1. Information Processing Systems 10 Servers 20 terminals 30 Microscope equipment

Claims

1. A method for training an estimator that estimates a moving direction of a focus of an objective lens relative to a focus position of a first object using a microscope equipped with the objective lens, the method comprising: an acquisition process in which a second object is photographed multiple times using the microscope equipped with the objective lens, with an aperture smaller than the maximum aperture, while changing the position of the objective lens in the optical axis direction relative to the second object at predetermined intervals, thereby acquiring at least two second microscope images; and a training data generation process in which a set of at least two divided second microscope images showing the same location of the second object acquired in the acquisition process is created. a learning process for performing machine learning on the images of the set of created second microscope images; A learning method that includes:

2. In the optical path, the diaphragm is located closer to the light source than the first object. The learning method according to claim 1 .

3. The teacher data generation process includes: calculating a relative position of the focal point of the objective lens with respect to a focus position of the second object in each of at least two of the second microscope images; The learning process includes: causing a learner to perform machine learning of training data including a combination of at least two of the second microscope images and the associated relative positions; The learning method according to claim 2 .

4. The teacher data generation process includes: calculating the relative position of the focal point of the objective lens with respect to the in-focus position of the second object in each of the second partial images; The plurality of second partial images and training data are generated by machine learning the learning device, In the combination, the associated relative position is selected as either one of the relative positions having a larger value or a smaller value among the relative positions corresponding to the plurality of second partial images. The learning method according to claim 3 .

5. In the acquisition process, Instead of using the diaphragm, a point light source is placed at the position of the diaphragm to acquire at least two of the second microscope images. The learning method according to claim 4.

6. The learning method according to any one of claims 1 to 5, A program that a computer runs.

Images