Method of autofocussing for imaging apparatus

Abstract

Provided is a computer-implemented method of autofocussing for an imaging apparatus. The method comprises: (a) receiving an image of an object from the imaging device; and (b) applying an image analysis model to the image to generate focus parameters of a target region of the object. The focus parameters provide: a focus quality of the target region in the image; and an indication of relative position of an object plane of the imaging device with respect to the target region of the object. The method further comprises (c) generating, using the focus parameters, a focus setting adjustment for the imaging device to improve the focus quality score.

Description

[0001] 008726812 1 Method of Autofocussing for Imaging ApparatusField of the InventionThe present invention relates to a method of autofocussing for an imaging apparatus, and to an imagingapparatus having an autofocus function.BackgroundIt is common for imaging devices (e.g. cameras, microscopes, telescopes and the like) to provide anautofocus function that enables an object in the field of view of the device to be automatically brought intofocus in a rapid manner. Autofocussing can be achieved in an active manner, e.g. by separatelydetecting a distance to an imaged object, or in a passive manner, e.g. by analysing data from images ofthe object.A common technique for passive autofocussing is based on contrast detection. For example, commercialmicroscopes may be supplied with passive image-based autofocus algorithms that use establishedmethods for computing image sharpness by summing the contrast between neighbouring pixels [11, 12,13, 14, 15, 16, 17]. Algorithms such as these rely on the presence of sharp edges surrounding theobjects to be focused. However, for small structures whose dimensions are close to the resolution limit ofthe microscope, diffraction fringes blur the edges, and this can cause traditional contrast-based autofocusalgorithms to fail. Moreover, in microstructures that exhibit some kind of periodicity, e.g. regular arrays ofmicro-pillars, interference between the diffraction patterns can create false sharp edges that compoundthe problem and are likely to cause failure of other sharpness measures, such as edge-based sharpness[18, 19, 20].Two previous studies have used deep regression models for image-based autofocusing in opticalmicroscopy [21, 22]. However, the scale of the structures imaged in these experiments and themagnifications used meant that the diffraction and interference effects discussed were not an issue.These experiments also required the equipment to be adapted by providing additional illumination of thesample by an off-axis LED, and furthermore required a pre-processing step to compute the Fouriertransform of the image.US 2022 / 0028116 A1 discloses a method of determining a focus position for an imaging device using atrained model. A first image is recorded at a first focus position and is used, along with a trained model, todetermine a second (improved) focus position. The method requires the images to contain an objectwhich is in the same context as that involved in training the model. The output of the model is a predictionof correct focus position, which in turn is used to drive actuators to cause the focus position of theAE9?AF? <=NA;= LG VBMEHW LG the predicted focus position.Another previous study utilized machine learning in an optical autofocus system for microscopy

[0023] .Here the impact of thermal or mechanical fluctuations over time in the optical autofocus system was 008726812 2addressed by training a convolutional neural network (CNN) over multiple days. The CNN was trainedusing a z-stack of images each with known degree of defocus, i.e. distance of imaged object from anobject plane of the imaging device, to generate a regression prediction of the defocus value. In use,captured images are resized and normalized to be suitable for input to the CNN. The output of the CNNis a prediction of the defocus in normalised units, which can then be rescaled to an actual defocus offsetwhich in turn can be used to adjust the z-stage of the microscope.Summary of the InventionAt its most general, the present invention provides a passive autofocus technique in which an artificialintelligence (AI) model is used to predict from an input image information about the extent to which atarget region of an object in that image is in focus (i.e., coincident with an object plane of an imagingdevice). The predicted information can be used to inform an adjustment of focus setting of an imagingdevice to improve the focus of the target region in a subsequently captured image. The AI modeldiscussed herein is configured to output two types of focus parameter. A first type relates to the focusquality of the image, e.g. a score that reflects how well focused the target region is in the image. Asecond type relates to the relative position of an object plane of the imaging device with respect to theobject being imaged, e.g. one or more scores that reflect the likelihood of the object plane being above orbelow the target region of the object. The output from the AI model may be used to control an autofocusfunction through a search routine that selects a focus setting adjustment. For example, the searchroutine may determine a focus setting adjustment that optimises the first focus parameter. The searchroutine may be adapted, e.g. weighted, based on the information conveyed by the second type of focusparameter. The combination of the AI model and search routine can provide a rapid autofocus techniquethat avoids disadvantages associated with contrast-based autofocus algorithms without the need todirectly predict a distance between the object and the object plane or perform additional imageprocessing (e.g. to compute a Fourier transform).According to a first aspect of the invention, there is provided a computer-implemented method ofautofocussing for an imaging device, the method comprising: (a) receiving an image of an object from theimaging device; (b) applying an image analysis model to the image to generate focus parameters of atarget region of the object, wherein the focus parameters provide: a focus quality of the target region inthe image; and an indication of relative position of an object plane of the imaging device with respect tothe target region of the object; and (c) generating, using the focus parameters, a focus setting adjustmentfor the imaging device to improve the focus quality.The method may be applied in any suitable imaging system in which a processor or other suitablecomputer-based controller is configured to adjust the focus settings of an imaging device. However, themethod may be particularly advantageous for use in imaging systems in which conventional autofocusalgorithms, such as those based on detecting differences in contrast, suffer due to problems withdiffraction and interference. For example, the method may be of particular benefit in the field ofmicroscopy, e.g. time lapse microscopy, for imaging microstructures that exhibit regular patterns or 008726812 3periodicity of some kind, i.e. repeating structure having dimensions of the order of 0.2-10 µm. Forexample, the method may be used in a traction force microscopy system to focus on the top surfaces ofmicro-pillars in a micro-pillar array.The method of the invention may also find use in inspecting of printed circuit boards (PCBs) or Micro-electromechanical Systems (MEMs) device, wherein the object is an upper surface of a PCB or MEMsdevice and the method may be used to focus on a region of the upper surface. In one example, theobject may be the upper surfaces of an array of MEMs devices, such as an array of micro-cantilevers,where the target region is an area of the upper surfaces for focussing on using the present method.The method of the invention may also find use in non-microscopic imaging systems where an object in animage may be noisy or blurred. Where traditional contrast-based auto-focus algorithms may fail, themethod of the present invention may be used to determine adjustment settings for such imaging systemsto focus on the object.The imaging device may thus be a microscope, e.g. with an adjustable z-stage. However, the methodmay also be applicable to image macroscopic objects, e.g. using a camera or telescope.The image analysis model, which may be referred to herein as an image processing model, or a trainedimage processing model, may be an AI model, such as a convolutional neural network (CNN) or a visualtransformer model (ViT). In other examples, the image analysis model may rely on classical imageanalysis methods (e.g., SITF). The image analysis model may receive an image as an input and mayoutput a vector (or a collection of scalar components) whose components represent the focus parametersdiscussed herein.The focus setting adjustment generated at step (c) of the method may be configured to instruct theimaging device to change the relative position of the object plane of the imaging device with respect tothe object (and therefore change the relative position of the object plane with respect to the target regionof the object). For example, it may be used to control or instruct movement of an adjustable z-stage in amicroscopy system or to instruct movement of an object lens of the imaging device.The method may further comprise: (d) applying the focus setting adjustment at the imaging device, andthen (e) capturing a further image of the object. For example, the method may comprise: iterativelyadjusting a focus setting of the imaging device (e.g. by repeating steps (a) to (e)) until an image in whichthe target region is determined to be optimally focused is obtained.In one example, the target region may be determined to be optimally focused if the focus quality providedby the focus parameter exceeds a predetermined threshold. Alternatively or additionally, the target regionmay be determined to be optimally focused if the focus setting adjustment indicates that a convergencecondition has been reached. To avoid endless iterations, the method may further comprise detecting ifthe focus setting adjustment indicates a non-convergence condition and, if so, terminating the method.As discussed above, the focus parameters provide two types of focus information. The first type relatesto focus quality. In one example, the focus quality is provided by outputting, from the image processingmodel, a probability score indicating a likelihood of the target region of the object being coincident with 008726812 4the object plane of the imaging device. For example, as the likelihood that the target region is coincidentwith the object plane increases, the focus quality probability score may tend to 1. In another example, the>G;MK IM9DALQ K;GJ= E9Q := :AF9JQ$ A%=% AF<A;9LAF? =AL@=J VAF->G;MKW GJ VFGL-in->G;MKW%The second type of focus information is an indication of relative position of the object plane of the imagingdevice with respect to the target region of the object. This focus information may be provided byoutputting, from the image processing model one or both of: (i) a Vbelow objectW probability scoreindicating a likelihood of the object plane of the imaging device being below the target region of theobject, and (ii) an Vabove objectW probability score indicating a likelihood of the object plane of the imagingdevice being above the target region of the object. .=J=AF$ L@= H@J9K= V:=DGO objectW may be used toindicate that the object plane of the imaging device is closer to the objective lens along its optical axisthan the target region of the object% 3AEAD9JDQ$ L@= H@J9K= V9:GN= objectW E9Q := MK=< LG AF<A;9L= L@9L L@=object plane of the imaging device is further from the objective lens along its optical axis than the targetregion of the object.Where the focus parameter comprises probability scores, the image analysis model may comprise aneural network (e.g. CNN) having a fully connected output layer configured to output the focusparameters and incorporating an activation function configured to determine probabilities. For example,the activation function may be a SoftMax activation function.Accordingly, in one example, the image processing model outputs three probability scores in response toa given input image, where the probability scores respectively quantify the likelihood that the object planeof the imaging device is at, above or below the target region of the object. It may be understood from thenature of these parameters that the output of the image processing model is not itself a prediction of adesired position for the object plane of the imaging device or even a prediction of the displacement (i.e.actual distance) between the target region of the object and the object plane of the imaging device.The step of generating the focus setting adjustment may comprise executing a search routine that isconfigured to optimise the focus quality of the target region by: receiving as an input the focus parametersand a current focus setting of the imaging device, and selecting an adjusted focus setting for the imagingapparatus that is predicted to correspond to an improved focus quality score. The search routine may forexample evaluate or extrapolate a predicted focus quality score for each of a plurality of candidate focussetting adjustments. The selected adjusted focus setting may correspond to the candidate focus settingwith the best predicted focus quality score. The method may thus be used to achieve autofocussing byusing the search routine to stepwise change the focus setting of the imaging device until an optimal focusis achieved.The search routine may comprise a Bayesian optimisation function that comprises a surrogate model andan acquisition function, the surrogate model being configured to map candidate focus settings of theimaging apparatus to corresponding predicted focus quality scores, and the acquisition function beingconfigured to select the adjusted focus setting from among the candidate focus settings. The surrogatemodel may be a Gaussian Process having a radial basis kernel function, and the acquisition function is anexpected improvement acquisition function configured to generate an expected improvement from a 008726812 5mean and a standard deviation value of the Gaussian Process. The search routine may be determined tohave reached a convergence condition if the expected improvement falls below a threshold. The searchroutine need not be limited to a Bayesian optimisation. Other converging search functions may also beused, e.g., genetic algorithms, stochastic hill climbing or gradient descent, grid search, etc.The method may include weighting the search routine using the indication of relative position of the targetregion with respect to the object plane of the imaging device. An advantage of this step is it can increasethroughput and facilitate rapid convergence of the search routine, which may be beneficial in scenarioswhere it is desirable to minimise the exposure of the object to the imaging process. The weighting step may be done for example by modifying the acquisition function. For example, the expected improvementacquisition function may be weighted using the indication of relative position of the object plane of theimaging device with respect to the target region of the object such that the expected improvementremains the same (or is increased) for candidate focus settings which match the indicated relativeposition and is reduced for candidate focus settings which do not match the indicated relative position. Inother words, if the output of the AI model indicates that the object plane is above the target region of theobject, the search routine will favour adjustments that take the object away from the objective lens soincrease the likelihood of aligning the object with the object plane. And vice versa if the output of the AImodel indicates that the object plane is below the object.Weighting the expected improvement acquisition function may be done using a weighting function, e.g. multiplying the output of the expected improvement acquisition function by a weighting function. The weighting function may take any suitable form that favours the relevant candidate focus setting adjustment as discussed above. For example, the weighting function may be a step function. For example, the weighting may be done by applying a step function defined by where is a probability score indicating a likelihood of the object plane of the imaging devicebeing below the object; is a probability score indicating a likelihood of the object plane of theimaging device being above the object; is a predetermined threshold probability score,represents a candidate position of an objective lens of the imaging device, and represents acurrent position of the objective lens corresponding to the current focus setting of the imaging device. Forexample, may be between 0.90 and 0.98, more preferably 0.95.As discussed above, the object may be disposed on an optical axis of the imaging device, and wherein adistance between the object and an objective lens of the imaging device along the optical axis may beadjustable to apply the focus setting adjustment. For example the distance may be adjusted (in the caseof the formula above) by moving an objective lens of the imaging device relative to the object. In other 008726812 6examples, the distance may be adjusted by moving the object relative to an objective lens of the imagingdevice (for example, by moving a stage that the object is positioned on).For example, the imaging device may include or may be part of an imaging system that includes acontroller that can drive an actuator to adjust the relative position of the object and the object plane of theimaging device based on the focus setting adjustment. The relative position of the object and the objectplane of the imaging device may be adjustable in a stepwise manner, e.g. using a stepper motor or servoactuator. The separation between candidate focus setting adjustments of the search routine maycorrespond with the scale of the stepwise adjustment that is achievable for the imaging device.The image processing model may be trained using a labelled training data set, the labelled training dataset comprising images of objects with target regions that are in focus and images of objects having targetregions which are not in focus.In another aspect, the present invention may provide a computer-implemented method of generatingtraining data for training an image processing model for use in autofocussing an imaging device, themethod comprising: receiving training images of an object, the training images being captured using animaging device, each training image being obtained at a respective displacement of the object planealong an optical axis of the imaging device; coating the target region in a fluorescent dye, receivingverification images of the object, the verification images having been captured using oil immersionconfocal microscopy at varying displacements of the object plane along the optical axis wherein the targetregion is coated in a fluorescent dye, determining an in-focus displacement along the optical axiscorresponding to a one of the verification images wherein the dyed target region is brightest, and labellingeach training image with a label that classifies the training image as in focus or not in-focus by comparingthe training images to the verification images.Alternatively the method of generating training data for training an image processing model for use inautofocussing an imaging device, may comprise: receiving training images of an object, the trainingimages being captured using an imaging device, each training image being obtained at a respectivedisplacement along an optical axis of the imaging device; labelling each training image with a label thatclassifies the training image as in focus or not in-focus based on whether a target region of the object is infocus in the training image; and verifying the labels by: coating the target region in a fluorescent dye,receiving verification images of the object, the verification images having been captured using oilimmersion confocal microscopy, the target region being coated in a fluorescent dye and images beingcaptured at varying displacements along the optical axis, determining an in-focus displacement along theoptical axis corresponding to a one of the verification images wherein the dyed target region is brightest,and verifying that the training images captured at the in-focus displacement are classified as in-focus.In a further aspect, there is provided an optical imaging system comprising: an imaging device forcapturing images of an object through an objective lens, the objective lens having a object plane locatedalong an optical axis; a stage for receiving the object to be imaged; an actuator configured to adjust adistance along the optical axis between the object plane and the target; and a controller configured to 008726812 7receive images from the imaging device and control the actuator using a focus setting adjustmentgenerated using the autofocussing technique outlined herein.The invention includes the combination of the aspects and preferred features described except wheresuch a combination is clearly impermissible or expressly avoided.Summary of the FiguresEmbodiments and experiments illustrating the principles of the invention will now be discussed withreference to the accompanying figures in which:Fig. 1 is a schematic diagram of an imaging apparatus suitable for use with embodiments of the presentinvention;Fig. 2 is a flowchart illustrating an exemplary process for autofocussing an imaging apparatus;Fig. 3 is a flowchart illustrating another exemplary process for autofocussing an imaging apparatus usinga search algorithm;Fig. 4 is a schematic diagram that depicts a convolutional neural network architecture for use in a methodof autofocussing an imaging apparatus;Fig. 5 is a flowchart illustrating an exemplary process for generating training data for training an imageprocessing model;Fig. 6 is a set of images showing a comparison of micropillar arrays captured in bright field mode (toprow) and fluorescent mode (bottom row) using confocal microscopy at various lens positions of themicroscope;Fig. 7A shows a set of graphs that depict the outputs of the convolutional neural network of Fig. 4 for avariety of input images at 100× magnification;Fig. 7B shows a set of graphs that depict the outputs of the convolutional neural network of Fig. 4 for avariety of input images at 60× magnification;Fig. 8A shows a set of graphs that compare sharpness scores for z-stacks of images captured at 100×magnification;Fig. 8B shows a set of graphs that compare sharpness scores for z-stacks of images captured at 60×magnification;Fig. 9 is a flowchart illustrating steps performed in an exemplary optimisation routine for autofocussing animaging apparatus;Figs. 10A-10D are a set of graphs that show stages of a Bayesian optimisation sequence that can beused in a method of autofocussing an imaging apparatus; and 008726812 8Fig. 11 shows (a) a multi-well plate with micro-pillars for use in traction force microscopy, (b) a magnifiedview of a portion of a micro-pillar array with 3 µm pitch, and (c) a magnified view of a portion of a micro-pillar array with 4 µm pitch.Detailed Description of the InventionAspects and embodiments of the present invention will now be discussed with reference to theaccompanying figures. Further aspects and embodiments will be apparent to those skilled in the art. Alldocuments mentioned in this text are incorporated herein by reference.A detailed discussion of the principles of the invention is given below in the context of imaging micro-pillararrays using traction-force microscopy to measure and track cellular forces as cells migrate acrossartificial surfaces designed to mimic the mechanical properties of biological tissues [1, 2, 3]. An importantapplication of this technique is the study of cancer cell proliferation and migration [4, 5]. At present thereis no high-content platform available for rapid testing of drugs that inhibit cancer cell migration, such asRho-Kinase inhibitor compounds [6, 7]. Most current methods rely on individual petri dishes with arrays ofmicropillars under high-end microscopes like the spinning disk Confocal microscope [8, 9]. The presentinvention may thus contribute to the provision of a multi-well platform capable of capturing time-lapseimages of cell migration across micropillar arrays in order to perform traction force microscopy for avariety of applications like drug screening.However, it is to be understood that the present invention need not be limited to the context of traction-force microscopy. The examples presented herein demonstrate that the invention is effective in thecontext of focussing on any type of micropatterned surface or microstructure exhibiting periodic features.The invention may be particularly effective where the scale of the structures being imaged is such that theeffects of diffraction and interference are an issue. For example, the technique disclosed herein may beused for quality assessment or process control of microcantilever manufacture, e.g. as part of an opticalmonitoring technique to check for unwanted buckling of microcantilever arms, e.g. within an array ofmicrocantilevers during fabrication.The embodiments discussed herein are based on a convolutional neural network (CNN) classifier thatpredicts whether a target region of an object (e.g. a target object of a type with which the classifier istrained) is in focus or out of focus in an input image. The CNN classifier presented herein requires noadditional illumination for image capture and needs no Fourier transform step during pre-processing. Inone example, the CNN classifier provides three output parameters. The three parameters are each aprobability that the object plane of the imaging device lies in a certain position with respect to the object tobe imaged. The first parameter is a probability that the object lies above the object. The secondparameter is a probability that the object lies coincident with the object (i.e. is in focus). The thirdparameter is a probability that the object lies below the object. As regression is not used in establishingthe model proposed herein, there is no direct prediction of the distance from the object plane to the targetregion of the object. Instead, it is proposed that the CNN classifier is used with a search routine to create 008726812 9an autofocus process for finding an optimal position for the object plane. Details of the search routine arediscussed in more detail below.The autofocus process discussed herein can be implemented on a conventional imaging device. Forexample, the autofocus process has been implemented by the inventors as a plugin for the an imagingsystem. Using this system, the inventors have tested the autofocus process on a petri dish withmicropillar arrays in an automatic image-capture mode with timelapse and also on a multi-well, highcontent platform comprising micro-pillar arrays. Testing was performed on bare pillar arrays and on pillararrays seeded with mouse pancreatic ductal adenocarcinoma (PDAC) cells and the results are discussedin detail below.System overviewFig. 1 is a schematic diagram of an imaging system 100 to which the autofocus function discussed hereincan be applied. The imaging system 100 comprises an imaging apparatus 120 and a controller 110 incommunication with the imaging apparatus 120 to control the relative position between an object plane ofthe imaging apparatus and an object 130 to be imaged. The imaging apparatus 120 may comprise animaging device 122 such as a microscope or the like configured to capture an image of the object 130 inthe field of view of an objective lens 124. The imaging apparatus 120 may further comprise a stage 126for holding the object 130. The imaging apparatus 120 is configured to permit relative movementbetween the stage 126 and an objective lens 124 of the imaging device to alter the position of the objectplane with respect to the object 130. For example, the objective lens 124 may be connected to actuatorsthat are configured to drive motion of the objective lens 124 relative to the object 126 along the opticalaxis 125 of the imaging device 122 (the up / down direction in the orientation shown in Fig. 1).In use, the imaging device 122 captures an image of the object 130 and communicates the capturedimage to the controller 110. The controller 110 includes a processor and memory having stored thereoncode, which when executed by the processor causes it to run an autofocus procedure. The autofocusprocedure uses as an input the captured image from the imaging device 122. As discussed in more detailbelow, the autofocus procedure may include running an autofocus algorithm that operates to classify theimage in terms of a position of the objective lens with respect to the object. The controller 110 uses theresult of this classification to generate adjustment instructions for the imaging device 122, e.g. to changethe relative distance between the objective lens 124 and object 130 to bring the object plane closer to theobject. The adjustment instructions may be generated using a search routine that takes the result of theclassification as an input. Following adjustment, the imaging device 122 may capture another image ofthe object 130. This second image may also be provided to the controller 110 to run another iteration ofthe autofocus process. The autofocus process may be iterated until a satisfactory image is obtained. 008726812 10Process overviewFig. 2 is a flowchart illustrating an exemplary process 200 for autofocussing an imaging device in line withthe discussion above. The process begins with a step S200 of receiving an image captured by theimaging device.The process continues with a step S202 of determining focus information from the captured image. Forexample, the focus information may be the result of inputting the captured image to a classificationalgorithm that operates to classify the image in terms of a position of the object plane with respect to thetarget region of the object. The classification algorithm may be a CNN classifier of the type discussedbelow. The focus information may include both (i) an indication of relative position of the object planewith respect to the target region of the object along the optical axis of the imaging device, and (ii) anindication of the quality of focus of the image. In one example, the indication of relative position may beprovided by a pair of scores that represent the likelihood that the object plane is above or below the targetregion of the object. Each of the pair of scores may be expressed as a probability. Similarly, theindication of the quality of focus of the image may be expressed as a probability that the object iscoincident with the object plane (which may be referred to as a focus quality or a focus quality score).The process continues with a step S204 of adjusting a focus setting of the imaging device based on thefocus information determined at step S202. Adjusting the focus setting may include changing the relativedistance between the object and the object plane of the imaging device. For example the object may bemounted on a stage that is movably coupled to the imaging device, or the objective lens may be movablealong an optical axis of the imaging device. The separation of the stage and the imaging device can thusbe controlled, and hence the distance between an object and the imaging device can be adjusted in acontrollable manner. In an example where the position of the objective lens is movable, if the focusinformation indicates that the object plane is beneath the object, the distance between the object and theobjective lens may be decreased. Where the focus information indicates that the object plane is abovethe object, the distance between the object and the objective lens may be increased. The amount bywhich the distance is changed may be selected based on the indication of the quality of focus. Forexample, where the quality of focus is low, a larger change may be selected.After the focus setting is adjusted, another image may be obtained.Fig. 3 is a flowchart illustrating another exemplary process 300 for autofocussing an imaging apparatususing a search algorithm. Similarly to the process 200 discussed above with respect to Fig. 2, thisprocess 300 begins with a step S300 of receiving an image captured by the imaging device and continueswith a step S302 of determining focus information from the captured image. The features of steps S200and S202 discussed above are also applicable to steps S300 and S302.The process continues with a step S304 of determining if the image (or a target region therein) is in focus,i.e. when the focus quality or a parameter of the search routine meets a predetermined focus criterion.For example, the image may be determined to be in focus when a maximum value of an ExpectedImprovement acquisition function (described below in relation to Fig. 9) is less than 0.05. 008726812 11If the determination in step S304 is yes, then the process continues with a step S312 of outputting theimage having the in-focus target region, following which the autofocus process terminates.If the determination in step S304 is no, then the process continues by selecting and applying a focusadjustment in order to obtain another image with improved focus properties. This part of the process mayuse a search routine to select a focus adjustment. In one example, this search routine may be aBayesian optimisation with a modified acquisition function that takes account of the focus information thatis determined in step S302. The process can therefore include a step S306 of adapting or otherwiseinitiating a search routine using the focus information. The process then includes a step S308 ofdetermining a focus adjustment using the search routine and a step S310 of adjusting the focus settingusing the determined focus adjustment. The features discussed above with respect to step S204 areapplicable to steps S308 and S310.Following adjustment, the process may cycle back to step S300 by obtaining another image of the objectwith the adjusted focus settings. The autofocus process may run iteratively until a satisfactory image isobtained (i.e. the determination in step S304 is yes) or a predetermined number of cycles is completed.As mentioned above, in one example the search routine used in the process of Fig. 3 is Bayesianoptimisation, which was originally developed as an efficient means of optimising an expensive black-boxfunction

[0026] . This is particularly suitable for imaging a micro-pillar array because in that context it isdesirable to minimise the number of images captured while searching for the optimal focus setting, inorder to minimise the light exposure of cells deposited on top of the pillar array. The Bayesianoptimisation used herein has a modified acquisition function that takes into account the focus informationdetermined by classifying the image. In particular, the modified acquisition function preferably makes useof a prediction of the direction of the object plane with respect to the target region of the object obtainedfrom the classification algorithm. This reduces the number of iterations required to achieve optimal focusby a factor of 2 to 3, compared with a conventional expected improvement (EI) acquisition function.Classification algorithmFig. 4 is a schematic diagram that depicts a convolutional neural network architecture for use in a methodof autofocussing an imaging apparatus. It is based on the well-known VGG-16 architecture

[0027] , but witha reduced number of convolutional (Conv) layers and max pooling (MaxPool) layers, and with half thenumber of filters per layer in order to speed up prediction without sacrificing accuracy. The finalconvolutional layer is converted to a vector by global average pooling and is followed by a single fullyconnected (FC) layer with 128 units. The model code is as follows, implemented with the Kerasframework: 008726812 12 The output layer consists of 3 units. A first unit classifies whether the image is focused on the pillar top. Asecond unit classifies whether the image lies below the pillar top (i.e. the object plane is below the objectto be imaged). A third unit classifies whether the image lies above the pillar top (i.e. the object plane isabove the object to be imaged). The second and third units may thus provide the pair of scores thatrepresent the likelihood that the object plane is above or below the object, as discussed above.Fig. 5 is a flowchart illustrating an exemplary process 500 for generating training data for training animage processing model. The process 500 begins with a step S500 of obtaining a data set.In experiments performed by the present inventors, the data set used to create the CNN classifierconsisted of 288 z-stacks comprising a total of 47000 images, of which 17000 were taken at 60×magnification and 30000 at 100× magnification. The z-stacks spanned a typical range of 10 to 50 µm,with a step size of 100 to 200 nm, although a few hundred images were also captured over a wider rangeof 2000 µm. Approximately half of the images contained cells, while the rest were captured using barepillars. In each of these two groups, there was an approximately even split between pillars with a 3 µmpitch and those with a 4 µm pitch. To ensure as robust a classifier as possible, a variety of differentsample conditions was included in the data set: (i) pillars stamped with a fresh silicon mould, resulting inflat pillar tops, (ii) pillars stamped with a previously used mould, resulting in rounded pillar tops, (iii) pillarsin MatTek dishes, (iv) pillars in a multi-well plate, (v) variable lighting intensity. The data set also includedexamples of collapsed pillars, pillars stuck together at the top, and small specks of debris.The process 500 continues with a step S502 of staining the pillar tops by stamping the array into a thinfilm of fluorescent dye (cellmask orange). The process 500 then continues with a step S504 of using oil 008726812 13immersion 100× objective confocal microscopy to capture z-stack verification images in both bright fieldand fluorescent imaging mode.Fig. 6 shows examples of such verification images. In Fig. 6 the bright field mode images are shown inthe top row and the fluorescent mode images are shown in the bottom row for various lens positions ,relative to the pillar tops at = 0. The original images have been cropped centrally for clarity. The brightfield image corresponding to the pillar tops occurs at the z-value where the fluorescent image is brightest.Using these verification images, the process 500 continues with a step S506 of determining the z-positionof a verification image with the brightest target region. This is determined to be the z-position where thetarget region is in-focus.Finally, in step S508 the training images are labelled as in-focus or not in-focus by matching their visualappearance to the verification images observed with the confocal microscope.In the above example, images within 200 nm of the pillar tops were labelled as focused (1). Theremainder were labelled either below (0) or above (2) the pillar tops. The raw images from themicroscopic imaging system were 16-bit TIFF files of dimensions 1536×2048 pixels, captured with a 12-bit camera. After labelling, these images were pre-processed by cropping centrally to an area of 256×256pixels and performing min-max normalisation. Many of the original images exhibited a slight gradient,resulting in height differences of ~400 µm across the width of the image. Centre-cropping reduced thisheight difference to ~50 µm and also reduced the time taken to train the network. Min-max normalisationensured that all images had an intensity range of 0T1 for input to the CNN.To evaluate the performance of the CNN the data set was split into a training set of 251 z-stacks (40000images) and a test set of 37 z-stacks (7000 images). The test set comprised 16 z-stacks (3200 images)captured at 60× objective magnification and 21 z-stacks (3800 images) at 100× objective magnification,and were chosen from the most recently captured data to be representative of live autofocus data. At thisstage the images were not shuffled, to ensure that images from different z-stacks were not intermingled.The training set was further divided into a training set and a validation set using an 80% / 20% stratifiedsplit to ensure an equal proportion of focused images in each set. At this stage the images were shuffled.Due to the high class imbalance (only 3% of images were focused), the minority class was augmented byreplicating all focused images in the training set in proportion to unfocused images. Data augmentationwas then applied, comprising random rotation of ±10°, random intensity variation of ±30% and randomzooming of ±25%.The CNN classifier was trained using the Keras framework on a GeForce RTX 2080 Ti graphicsprocessing unit (GPU). Using stochastic gradient descent with momentum = 0.9 and a batch size of 64,the model took 14 hours to train for 200 epochs. The validation set was used to determine the optimumlearning rate of 0.01, as well as to experiment with different model architectures and to evaluateregularisation methods such as dropout and % 4@= E=LJA; MK=< LG BM<?= L@= EG<=DXK H=J>GJE9F;= O9Kthe F1-score, which takes into account class imbalance in the validation and test sets. 008726812 14Regularisation was not found to improve performance. The optimum model architecture was found to bea 7-layer CNN, followed by a single FC layer, as shown in Fig. 4. Prediction speed on the centralprocessing unit (CPU) increased by a factor of 5 from 400 ms to 80 ms per image when the number ofconvolutional layers was reduced from 10 to 7. Reducing the number of convolutional layers furtherresulted in poorer performance, as measured by the F1-score, as well as a longer training time, while theaddition of extra FC layers did not improve performance.For the validation set, a value of F1 = 0.98 was obtained. For the test set, a slightly lower value of F1 =0.95 was obtained, indicating that the model slightly overfits the validation set. The confusion matrix isshown in Table 1.Classified Below Focused Above Below 3227 18 80 Labelled Focused 20 119 20 Above 188 25 3307 Table 1: Confusion matrix for test setApproximately 25% of the focused images are misclassified, compared to 3% of images below the pillartops and 6% of images above. The large proportion of misclassifications amongst the focused images isnot a major cause for concern as most lie close to the optimal focus and in fact reflect inaccuracies in thelabelling of the raw data.The misclassifications of images above / below the pillar tops are a little more problematic, but in thediscussion of the search routine below, it is shown that the Bayesian optimisation parameters can bechosen to ensure that the autofocus algorithm finds the optimal focus position with a high degree ofaccuracy despite the misclassifications.Figs. 7A and 7B show the predictive distributions of the CNN for several z-stacks in the test set. Thefocus probability is a sharp function that peaks very close to the labelled optimal focus position. Fig. 7Ashows four graphs, each corresponding to a z-stack obtained at 100× magnification. Fig. 7B shows foursimilar graphs corresponding to z-stacks obtained at 60× magnification. Graphs (a) and (e) relate to abare pillar array with 3 µm pitch, graphs (b) and (f) relate to a bare pillar array with 4 µm pitch, graphs (c)and (g) relate to a pillar array with 3 µm pitch having cells thereon, and graphs (d) and (h) relate to a pillararray with 4 µm pitch having cells thereon. The x-axis of each graph corresponds to the z dimension, i.e.direction along the optical axis, with the pillar top (i.e. in-focus) position at = 0. The location of the pillarbases is denoted by vertical line 700. In each graph, a line indicating the probability of the object planebeing coincident with the pillar tops comprises a peak at the pillar top position with no significantmisclassifications far from the pillar top position. 008726812 15The model predicts less accurately for whether the image is below or above the pillar tops, especially atz-values > 8 µm from the optimal focus position. However, the probability of being above / below theoptimal focus position rises / falls sharply for these parameters as z is scanned through the optimal focusposition. The range of z over which the classifier can be relied upon is 8 µm (a conservative estimate).Outside of this range the classifier cannot be relied upon to discriminate strongly between images that lieabove or below the optimal focus position.The performance of the classifier may be compared with the performance of contrast-based sharpnessscores for similar images. Three of the most widely used sharpness scores are the Brenner gradient where is the pixel intensity at row , column of the image, is the number of rows, is thenumber of columns and is the mean pixel intensity across the whole image.Figs. 8A and 8B show a set of graphs that compare sharpness scores for z-stacks of images acquired byscanning the lens position along the optical axis and through the optimal focus position in steps of 100nm at 100× magnification and 60× magnification. Similar to Figs. 7A and 7B, graphs (a) and (e) relate toa bare pillar array with 3 µm pitch, graphs (b) and (f) relate to a bare pillar array with 4 µm pitch, graphs(c) and (g) relate to a pillar array with 3 µm pitch having cells thereon, and graphs (d) and (h) relate to apillar array with 4 µm pitch having cells thereon. The x-axis of each graph corresponds to the zdimension. A small range of z (± 200 nm) over which the image can be judged to be optimally focused byan expert is indicated at = 0. The likely location of the pillar bases is denoted by vertical line 800, basedon the specified pillar height from the fabrication process.The Brenner gradient 802 9F< 5GDD9L@XK 804 and 806 were calculated for each z-stack and areplotted on the graphs.None of the sharpness scores peak reliably when the object plane of the imaging device is coincident withthe pillar tops. All scores exhibit a pronounced peak at the z-position corresponding to the pillar bases9F< 9 K=;GF<9JQ H=9C L@9L DA=K ;DGK= LG L@= HADD9J LGHK% -GJ L@= *J=FF=J ?J9<A=FL 9F< 5GDD9L@XK , thesecondary peak is roughly coincident with the pillar tops when the pitch is 4 µm, but is systematicallyK@A>L=< 9:GN= L@= HADD9J LGHK O@=F L@= HAL;@ AK ) SE% 4@= K=;GF<9JQ H=9C AF 5GDD9L@XK does notcorrespond to the pillar tops for either pitch, and sometimes dominates the peak at the pillar bases (seee.g. graphs (b) and (h)).The lack of a single peak at the pillar tops in any of the sharpness scores makes sharpness an unsuitablequantity by which to judge focus quality for traction force microscopy. While it is tempting to determine theposition of the pillar tops by first locating the primary peak at the pillar bases, then adjusting the focus 008726812 16upwards by a fixed amount, this is undesirable as the pillar height tends to reduce with each re-use of thesilicon mould, introducing uncertainty in the pillar height. There is also a risk that a peak-finding algorithmwould find the wrong peak. The sharpness function would need to be sampled at a large number ofpositions to avoid such an error, and this is incompatible with the need to minimise light exposure of cellsduring autofocusing.Search routineAs discussed above with reference to Fig. 3, the autofocus process discussed herein uses the focusinformation output by the classifier in a search routine to decide how far to move the lens, and in whichdirection, to reach the optimal focus in a small number of steps.Whilst the search routine could be implemented using a global search strategy such as a brute forcesearch of stochastic hill climbing, such a solution would require a large number of image samples, whichin the micro-pillar imaging context discussed herein is undesirable because it would expose cells to toomuch light.The preferred search routine adopted herein is a Bayesian optimisation with a modified acquisitionfunction that takes into account the predictions from the image analysis model of the direction of the lensposition with respect to the object. The steps of the search routine are depicted in the flow chart of Fig. 9.The advantage of using Bayesian optimisation over a simple binary search is that binary search on itsown would be reliable only in regions where the classification probabilities for the direction of the objectiveplane with respect to the object were close to 1. It would be useful only for narrowing the search space toa region within ± 1 µm of the optimal focus position. A fine step-wise search would then be needed to findthe optimum.In the approach disclosed herein, the search routine instead uses the CNN predictions to down-weightthe acquisition function. Rather than completely ruling out the regions below / above the sampling location,this simply reduces the probability that they will be sampled, yielding a search strategy that is similar tobinary search, but more robust, and that is capable of locating the optimal focus position without the needfor a fine step-wise search.The search routine begins with a step S1000 of receiving the output from the classification of the image,which in this case is the output of the CNN classifier that indicates focus quality (i.e. probability that atarget object is in focus) and relative position of the object plane with respect to the object (i.e. aprobability that object plane of the imaging device is above the object and a probability that the objectplane is below the object T these predicted probabilities are referred to as ).The Bayesian optimisation of the focus probability is then implemented in step S1002 using a GaussianProcess (GP) with a radial basis function kernel, weighted by a constant : 008726812 17where is the distance between and , which is simply in our one-dimensional case, andis a parameter known as the length scale, which determines the smoothness of the GP.For the acquisition function, the Expected Improvement ( ) was chosen Uwell known for balancingexploration vs exploitationUand was weighted with a step function. The search routine includes a stepS1004 of evaluating, for each successive sample , the CNN predicted position probability, and, if it exceeds a threshold of 0.95, weighting the with a step function taking avalue of 0.3 for all z-values below / above and a value of 1.0 for all z-values above / below the current lensposition. This reduces the EI in regions unlikely to contain the optimal focus position, thus increasing theprobability of subsequent sampling in a region likely to contain the optimal focus position. where , are the mean and standard deviation of the GP, is the maximum value of ,, are the probability density function and cumulative distribution function of the normaldistribution for a random variable , and With the configuration above, the search routine continues with a step S1006 of determining adjustedfocus settings (e.g. an adjusted value of that can be used to change the relative distance between theobject and object plane of the imaging device) from the output of the GP. .4@= HJAGJ Z"R# = 0 was used.The kernel parameters h and lambda and the noise parameter of the GP were optimised by observing thequality of the fit visually. The set of z-values over which the EI was calculated was restricted to a range of30 µm and a step size z-step = 100 nm (the minimum step size of the motor that drives a lens in theimaging system). A length scale ( = 600 nm) was chosen well in excess of the step size to ensure ahigh degree of smoothness in the GP. The rationale for this choice was to force the maximum of the fittedGP to reside at the centre of the range of z-values classified as focused. For the same reason, a highvalue for the noise parameter was chosen wherein = 0.1. The quality of the fit was less sensitive to theparameter , but a value of = 2 produced good results.In setting a convergence criterion for the Bayesian optimisation process, there were two context-specificrequirements: (1) the need to minimise the number of z-values sampled, to ensure the cells were not exposed to anexcessive amount of light,(2) the importance of locating the optimal focal position accurately to minimise subsequentsystematic errors in the traction force. 008726812 18To satisfy the former a maximum iteration count was set to 20. For the latter stopping criterion waschosen based on the maximum value of the : stop when < 0.05. With the GP parameter valuesdescribed above, a robust, reliable autofocus was achieved that converged within 13-14 iterations over aduration of 6 seconds. These values were consistent for both 60× and 100× magnification and for pitchesof 3 µm and 4 µm.Figs. 10A to 10B are a set of graphs that show stages of a Bayesian optimisation sequence that can beused in a method of autofocussing an imaging apparatus. These graphs show the final GP fitted to atypical converged sequence of Bayesian optimisation. This sequence was captured at 100× objectivemagnification using a pillar array with pitch 4 µm with PDAC cells deposited on top. The Bayesianoptimisation sequences for all combinations of 60×, 100× objective magnification and pitch 3, 4 µm werefound to be similar.The inset to Fig. 10D shows the fit that can be achieved with a shorter length scale = 200 nm. The GPfollows the data more closely, but is also more sensitive to noise in the CNN predictions, which risks mis-identifying the focal position. A longer length scale (600 nm in the example above) reduces the quality ofthe fit but improves the convergence of Bayesian optimisation to the true focal positionExamplesIn the examples discussed herein, the imaging apparatus 120 was a standard commercial tabletopmicroscope and the object 130 is an array of micro-pillars. The controller 110 is configured to provide theimaging apparatus 120 with an autofocus function that can compensate for focal drift between differentobjects and over time. The autofocus function is used to ensure that the top of the pillar array is in focusto enable accurate measurement of cellular forces, which are proportional to the displacements of thepillar tops

[0010] .Pillar fabrication methods were adapted from

[0024] . Briefly, an elastomeric material (for example,Polydimethylsiloxane (PDMS; Sylgard 184, Dow Corning)) was mixed at a 10:1 elastomer base:curingagent ratio, degassed, and placed onto 35-mm glass bottom tissue culture dishes (MatTek). Silicontemplates with hole structures were placed pattern side down onto the PDMS and allowed to cure at 70°C>GJ '( @GMJK$ ?ANAF? 1+03 OAL@ ( 019 7GMF?XK EG<MDMK% 3ADA;GF L=EHD9L=K O=J= L@=F J=EGN=< >JGE L@=cured PDMS while immersed in absolute ethanol.PDMS pillar substrates were coated with human fibronectin (R&D Systems) at 10 mg / mL finalconcentration for 1 hour at 37°C. The fluorescent staining of the pillar top with cell mask orange (ThermoFisher C10045) was done via microcontact printing

[0025] , briefly the cured PDMS pillars were oxygenplasma treated (80 W, 1 min) and the pillar tops were gently stamped using another stained PDMS blockfor 1-5 sec. PDAC cells were seeded at 5,000 cells / cm2 in ;GEHD=L= +0,0 " +MD:=;;GXK EGA=< ,9?D=XKmedium) and allowed to attach overnight.Frame (a) in Fig. 11 shows a multi-well plate with micropillars for use in traction force microscopy. Themicropillars are of diameter approximately 1 µm and height 2.8 µm and are arranged in square arrays. 008726812 19The centre-to-centre pitch is either 3 µm or 4 µm, these values being chosen as appropriate for the celltypes to be studied. Frame (b) shows a magnified view of a portion of a micropillar array with 3 µm pitch,and frame (c) shows a magnified view of a portion of a micro-pillar array with 4 µm pitch.The images discussed in the examples disclosed herein were obtained by timelapse light microscopyusing a microscopic imaging system with an on-stage incubator (Thermo Fisher Scientific) operating atstandard cell culture conditions of 37°C and 5% CO2. Single-cell images were captured every 1 minutesfor 1 hour using a 100× and 60× air objective with 0.95 and 0.8 numerical aperture respectively(Olympus).The autofocus function discussed above was tested on a newly prepared MatTek dish and multi-wellplates containing pillars of 3 µm pitch in some and pillars of 4 µm pitch in others. All dishes containedPDAC cells.To evaluate the reliability at a single point we ran the autofocus process multiple times from the samestarting position. Table 2 shows the results of this test for four different conditions: 60×, 100× objectivemagnification and 3 µm, 4 µm pitch.Objective Pitch Test runs Success rate Average iterations Magnification 60× 3 µm 25 100% 14 -3.7 µm 200 nm 4 µm 53 94% 14 -2.4 µm 230 nm 100× 3 µm 24 100% 15 -6.8 µm 150 nm 4 µm 50 100% 15 -5.5 µm 200 nm Table 2: Reliability of autofocus plugin during multiple runs from the same starting position.The success rate is the proportion of runs that converged close to the optimal focus position. The tableshows the average number of iterations required for convergence in successful runs, also the meandistance from the starting position to the converged location of the optimal focus position and thestandard deviation of these distances.The process obtained a high rate of successful convergence (100% for 3 of the conditions and 94% forthe other). Of those runs that converged successfully, the average number of iterations was 14 or 15,taking an average of 5 s to complete. The distance of the starting position from the optimal focusposition was slightly different in each of the test conditions, but the standard deviation over all runswithin each condition was fairly similar: around 200 nm, which is within the tolerance of 300 nm in ouroriginal specifications.Of those few runs that failed to converge, the reason for failure was a mis-classification of the direction ofthe object plane in the first few points of the Bayesian optimisation sequence before the main peak hadbeen discovered. This falsely lowered the EI in the region of the object plane to such an extent that future 008726812 20points were not sampled in that region. If the misclassification occurred later in the sequence, after themain peak had already been identified, the effect on the EI was less drastic and did not prevent futurepoints being sampled in the region of the optimum.The robustness of the autofocus algorithm was further tested by capturing two time-lapse sequences, oneat 60× objective magnification and the other at 100× objective magnification. Each sequence traversedtwo wells of a multi-well high content platform per time interval of 2 minutes, one containing pillars of 3 µmpitch and other 4 µm pitch. Each well was sampled in two different locations, yielding a total of fourimages per time interval. The total duration of the time-lapse sequence was 1 hour. During that time, theobject plane drifted by 2 to 5 µm (we observed different, apparently random, amounts of drift in differentwells and at different magnifications). The autofocus algorithm was used for every image captured.During the sequence at 100× objective magnification, not a single autofocus failed. For the sequence at60× magnification, the autofocus failed 5 times out of 120. Failures occurred both early and late in thetime-lapse sequence, and were not caused by drift of the starting position away from the optimal focusposition Only the well containing pillars of 4 µm pitch was affected. At one of the positions within this wella cell was partially visible in the central region; at the other position no cell was visible. The positionwithout the cell experienced 4 out of 5 of the failures.The autofocus routine was also tested for robustness to defects in the pillar arrays such as collapsedpillars, small pieces of debris / bacteria etc. The autofocus process coped well at both 60× and 100×objective magnification, failing only for very large numbers of collapsed pillars in the central region andlarge pieces of debris covering a substantial portion of the central region.Accurate measurement of cellular forces relies on the ability of the microscope to focus accurately on thetop of the pillar array, as the pillar displacements are proportional to the forces exerted by the cellaccording to the equation:where is the lateral traction force on the pillar top, is a correction factor for taking substratebending into account, is the lateral displacement of the pillar top perpendicular to the pillar axis, andis a spring constant given by: where AK L@= 7GMF?XK EG<MDMK G> L@= HGDQE=J "1+03 AF L@= HJ=K=FL =P9EHD=K#$ is the pillar diameter,and is the pillar height

[0010] .Pillar bending is dependent only on the aspect ratio of the pillars and the Poisson ratio (transversecompression to axial strain under uniaxial loading) U parameters that are fixed during the fabricationprocess. Hence, the traction force is directly proportional to the lateral displacement of the pillar top.The fractional uncertainty in the traction force is therefore the same as the fractional uncertainty in thedisplacement of the pillar top, which is equal to the fractional uncertainty in the pillar height, by the law ofsimilar triangles: 008726812 21where is the uncertainty in the traction force, is the uncertainty in the lateral displacement ofthe pillar top and is the uncertainty in the z-value of the pillar top. From the results of the experimentshown in Table 2, the largest value of was 230 nm. Since the pillars were fabricated with height =3 µm, the autofocus process discussed herein gives rise to a fractional error in the traction force of 8%.The features disclosed in the foregoing description, or in the following claims, or in the accompanyingdrawings, expressed in their specific forms or in terms of a means for performing the disclosed function,or a method or process for obtaining the disclosed results, as appropriate, may, separately, or in anycombination of such features, be utilised for realising the invention in diverse forms thereof.While the invention has been described in conjunction with the exemplary embodiments described above,many equivalent modifications and variations will be apparent to those skilled in the art when given thisdisclosure. Accordingly, the exemplary embodiments of the invention set forth above are considered tobe illustrative and not limiting. Various changes to the described embodiments may be made withoutdeparting from the spirit and scope of the invention.For the avoidance of any doubt, any theoretical explanations provided herein are provided for thepurposes of improving the understanding of a reader. The inventors do not wish to be bound by any ofthese theoretical explanations.Any section headings used herein are for organizational purposes only and are not to be construed aslimiting the subject matter described.Throughout this specification, including the claims which follow, unless the context requires otherwise, theOGJ< V;GEHJAK=W 9F< VAF;DM<=W$ 9F< N9JA9LAGFK KM;@ 9K V;GEHJAK=KW$ V;GEHJAKAF?W$ 9F< VAF;DM<AF?W OADD :=understood to imply the inclusion of a stated integer or step or group of integers or steps but not theexclusion of any other integer or step or group of integers or steps. / L EMKL := FGL=< L@9L$ 9K MK=< AF L@= KH=;A>A;9LAGF 9F< L@= 9HH=F<=< ;D9AEK$ L@= KAF?MD9J >GJEK V9$W V9F$W9F< VL@=W AF;DM<= HDMJ9D J=>=J=FLK MFD=KK L@= ;GFL=PL ;D=9JDQ <A;L9L=K GL@=JOAK=% 29F?=K E9Q := =PHJ=KK=<@=J=AF 9K >JGE V9:GMLW GF= H9JLA;MD9J N9DM=$ 9F<&GJ LG V9:GMLW 9FGL@=J H9JLA;MD9J N9DM=% 6@=F KM;@ 9 J9F?=is expressed, another embodiment includes from the one particular value and / or to the other particularvalue. Similarly, when values are expressed as 9HHJGPAE9LAGFK$ :Q L@= MK= G> L@= 9FL=;=<=FL V9:GML$W ALOADD := MF<=JKLGG< L@9L L@= H9JLA;MD9J N9DM= >GJEK 9FGL@=J =E:G<AE=FL% 4@= L=JE V9:GMLW AF J=D9LAGF LG 9numerical value is optional and means for example + / - 10%.ReferencesA number of publications are cited above in order to more fully describe and disclose the invention andthe state of the art to which the invention pertains. Full citations for these references are provided below.The entirety of each of these references is incorporated herein. 008726812 22[1] Tan JL, Tien J, Pirone DM, Gray DS, Bhadriraju K, Chen CS. Cells lying on a bed ofmicroneedles: An approach to isolate mechanical force. Proceedings of the National Academy ofSciences 2003;100(4):1484T1489.[2] du Roure O, Saez A, Buguin A, Austin RH, Chavrier P, Silberzan P, et al. Force mapping inepithelial cell migration. Proceedings of the National Academy of Sciences 2005;102(7):23902395.[3] Ermis M, Antmen E, Hasirci V. Micro and Nanofabrication methods to control cell-substrateinteractions and cell behavior: A review from the tissue engineering perspective. BioactiveMaterials 2018;3:355T369.[4] Franz CM, Jones GE, Ridley AJ. Cell Migrationin Development and Disease. Developmental Cell2002;2:153T158.[5] Chaudhuri PK, Pan CQ, Low BC, Lim CT. Topography induces differential sensitivity on cancercell proliferation via Rho-ROCK-Myosin contractility. Scientific Reports 2016;6:19672.[6] Wei L, Surma M, Shi S, Lambert-Cheatham N, Shi J. Novel Insights into the Roles of Rho Kinasein Cancer. Arch Immunol Ther Exp 2016;64:259T278.[7] Rath N, Munro J, Cutiongco MF, Jagi=YYo A, Gadegaard N, McGarry L, et al. Rho KinaseInhibition by AT13148 Blocks Pancreatic Ductal Adenocarcinoma Invasion and Tumor Growth.Cancer Research 2018;78(12):3321T3336.[8] van Hoorn, H., Harkes, R., Spiesz, E.M., Storm, C., van Noort, D., Ladoux, B. and Schmidt, T.,2014. The nanoscale architecture of force-bearing focal adhesions. Nano letters, 14(8), pp.4257-4262.[9] Jin, B.J., Lee, S. and Verkman, A.S., 2018. Hollow micropillar array method for high-capacitydrug screening on filter-grown epithelial cells. Analytical chemistry, 90(12), pp.7675-7681.

[0010] Schoen I, Hu W, Klotzsch E, Vogel V. Probing Cellular Traction Forces by Micropillar Arrays:Contribution of Substrate Warping to Pillar Deflection. Nano Letters 2010;10(5):1823T1830.

[0011] Muller RA, Buffington A. Real-time correction of atmospherically degraded telescope imagesthrough image sharpening. Journal of the Optical Society of America 1974;64(9):1200T1210.

[0012] Brenner JF, Dew BS, Horton JB, King T, Neurath PW, Selles WD. An automated microscope forcytologic research A preliminary evaluation. Journal of Histochemistry and Cytochemistry1976;24(1):100T111.

[0013] Erteza A. Sharpness index and its application to focus control. Applied Optics 1976;15(4):877T881.

[0014] Erteza A. Depth of convergence of a sharpness index autofocus system. Applied Optics1977;16(8):2273T2278.

[0015] Vollath D. The influence of the scene parameters and of noise on the behaviour of automaticfocusing algorithms. Journal of microscopy 1988;151(2):133T146.

[0016] Groen FC, Young IT, Ligthart G. A comparison of different focus functions for use in autofocusalgorithms. Cytometry: The Journal of the International Society for Analytical Cytology1985;6(2):81T91.

[0017] Yazdanfar S, Kenny KB, Tasimi K, Corwin AD, Dixon EL, Filkins RJ. Simple and robust image-based autofocusing for digital microscopy. Optics express 2008;16(12):8670T8677. 008726812 23

[0018] Caviedes J, Gurbuz S. No-reference sharpness metric based on local edge kurtosis. In:Proceedings. International Conference on Image Processing, vol. 3 IEEE; 2002. p. IIITIII.

[0019] Dijk J, Van Ginkel M, Van Asselt RJ, Van Vliet LJ, Verbeek PW. A new sharpness measurebased on Gaussian lines and edges. In: International Conference on Computer Analysis ofImages and Patterns Springer; 2003. p. 149T156.

[0020] Yao Y, Abidi B, Doggaz N, Abidi M. Evaluation of sharpness measures and search algorithms forthe auto-focusing of high-magnification images. In: Visual Information Processing XV, vol. 6246International Society for Optics and Photonics; 2006. p. 62460G.

[0021] Jiang S, Liao J, Bian Z, Guo K, Zhang Y, Zheng G. Transform- and multi-domain deep learningfor single-frame rapid autofocusing in whole slide imaging. Biomedical Optics Express2018;9(4):1601T1612.

[0022] Pinkard H, Phillips Z, Babakhani A, Fletcher DA, Waller L. Deep learning for single-shotautofocus microscopy. Optica 2019;6(6):794T797.

[0023] Lightley, J., Görlitz, F., Kumar, S. et al. (2022). Robust deep learning optical autofocus systemapplied to automated multiwell plate single molecule localization microscopy. J. Microsc, 288,130T 141. https: / / doi.org / 10.1111 / jmi.13020

[0024] Ghassemi S, Meacci G, Liu S, Gondarenkoa AA, Mathur A, Roca-Cusachs P, et al. Cells testsubstrate rigidity by local contractions on submicrometer pillars. Proceedings of the NationalAcademy of Sciences 2012;109(14):5328U-5333.

[0025] Sunghoon Joo YN Kyungtae Kang. In vitro neurite guidance effects induced by polylysinepinstripe micropatterns with polylysine background. J Biomed Mater Res Part A2015;103A:2731T2739.

[0026] Mockus J. Bayesian Approach to Global Optimization: Theory and Applications. Springer; 1989.

[0027] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition.arXiv:14091556 2015.

Claims

008726812 24 Claims 1. A computer-implemented method of autofocussing for an imaging device, the methodcomprising: (a) receiving an image of an object from the imaging device;(b) applying an image analysis model to the image to generate focus parameters of a targetregion of the object, wherein the focus parameters provide:a focus quality of the target region in the image; andan indication of relative position of an object plane of the imaging device with respect tothe target region of the object; and(c) generating, using the focus parameters, a focus setting adjustment for the imaging device toimprove the focus quality.

2. The method of claim 1, wherein the focus setting adjustment is configured to instruct theimaging device to change the relative position of the object plane of the imaging device with respect tothe target region of the object.

3. The method of claim 1 or 2 further comprising:(d) applying the focus setting adjustment at the imaging device, and then(e) capturing a further image of the object.

4. The method of claim 3 further comprising repeating steps (a) to (e) until the target regionis determined to be in focus.

5. The method of any preceding claim, wherein the image analysis model is a convolutionalneural network, CNN.

6. The method of any preceding claim, wherein the focus parameters provide the focusquality by outputting, from the image analysis model, a probability score indicating a likelihood of thetarget region being coincident with the object plane of the imaging device.

7. The method of any preceding claim, wherein the focus parameters provide the indicationof relative position of the object plane of the imaging device with respect to the target region of the objectby outputting, from the image analysis model:a below object probability score indicating a likelihood of the object plane of the imaging devicebeing below the target region of the object, and / oran above object probability score indicating a likelihood of the object plane of the imaging devicebeing above the target region of the object.008726812 25 8. The method of any preceding claim, wherein generating the focus setting adjustmentcomprises executing a search routine that is configured to optimise the focus quality of the target regionby: receiving, as inputs, the focus parameters and a current focus setting of the imaging device, andselecting an adjusted focus setting for the imaging apparatus that is predicted to correspond to an improved focus quality score.

9. The method of claim 8, wherein the search routine comprises a Bayesian optimisationfunction that comprises a surrogate model and an acquisition function, the surrogate model beingconfigured to map candidate focus settings of the imaging apparatus to corresponding predicted focusquality scores, and the acquisition function being configured to select the adjusted focus setting fromamong the candidate focus settings.

10. The method of claim 9, wherein the surrogate model is a Gaussian Process having aradial basis kernel function, and the acquisition function is an expected improvement acquisition functionconfigured to generate an expected improvement from a mean and a standard deviation value of the Gaussian Process.

11. The method of any one of claims 8 to 10 further comprising weighting the search routineusing the indication of relative position of the object plane of the imaging device with respect to the targetregion of the object.

12. The method of claim 10 further comprising weighting the expected improvementacquisition function using the indication of relative position of the object plane of the imaging device withrespect to the target region of the object such that the expected improvement is reduced for candidatefocus settings which do not match the indicated relative position and remains the same for candidatefocus settings which match the indicated relative position.

13. The method of claim 12, wherein weighting the expected improvement acquisitionfunction comprises applying a step function defined by:wherein: is a below object probability score indicating a likelihood of the object plane of theimaging device being below the target region of the object; is an above object probability score indicating a likelihood of the object plane of theimaging device being above the target region of the object; is a predetermined threshold probability score;008726812 26 represents a candidate focus position of the target region, andrepresents a current focus position of the target region corresponding to the current focussetting of the imaging device.

14. The method of any preceding claim, wherein the object is disposed on an optical axis ofthe imaging device, and wherein a distance between the object and an objective lens of the imagingdevice along the optical axis is adjustable to apply the focus setting adjustment.

15. The method of any preceding claim, wherein the image processing model is trained usinga labelled training data set, the labelled training data set comprising images of objects with target regionsthat are in focus and images of objects having target regions which are not in focus.

16. The method of any preceding claim, wherein the target region of the object comprises amicrostructure.

17. The method of claim 16, wherein the microstructure comprises a periodic pattern.

18. A computer-implemented method of generating training data for training an imageprocessing model for use in autofocussing an imaging device, the method comprising:receiving training images of an object, the training images being captured using an imagingdevice, each training image being obtained at a respective displacement of the object plane along anoptical axis of the imaging device;coating the target region in a fluorescent dye,receiving verification images of the object, the verification images having been captured using oilimmersion confocal microscopy, at varying object plane displacements along the optical axis, wherein thetarget region is coated in a fluorescent dye,determining an in-focus displacement along the optical axis corresponding to a one of theverification images wherein the dyed target region is brightest, andlabelling each training image with a label that classifies the training image as in focus or not in-focus by comparing the training images to the verification images.

19. An optical imaging system comprising:an imaging device for capturing images of an object through an objective lens, the objective lenshaving an object plane located along an optical axis;a stage for receiving the object to be imaged;an actuator configured to adjust a distance along the optical axis between the object plane of theimaging device and the target region of the object; anda controller configured to receive images from the imaging device and control the actuator usinga focus setting adjustment generated by the method of any one of claims 1 to 17.

Images