Classifying liquid handling procedures using neural networks

Abstract

A method for classifying a liquid handling procedure includes, such as a pipetting procedure occurring during a laboratory automated analysis: receiving measurement data (56) encoding a measurement curve (28, 30, 32) of a measurement over time during at least a portion of the liquid handling procedure; inputting the measurement data (56) into a neural network (57); and calculating at least one quality value (74, 86) for the liquid handling procedure with the neural network (57). The neural network allows classifying the liquid handling procedure as normal or abnormal, e.g. a blockage in a line, presence of air or bubbles, etc.

Description

TECHNICAL FIELD

[0001] The present invention relates to a method, a computer program and a computer readable medium for classifying a liquid handling procedure, and to a laboratory automation system. BACKGROUND

[0002] Laboratory automation systems are used to automate the tasks of a laboratory assistant, e.g. testing a patient for a certain disease. Typically, samples of a patient’s blood, urine, stool are collected and analyzed by biochemical procedures. Such procedures comprise various operations like adding substances, incubation, separation, etc. as well as measurement processes which quantitatively or qualitatively measure the amount or presence of substances indicative of a certain disease.

[0003] An important part of a laboratory automation system is a liquid handling system which typically comprises one or more pipettes which can be moved in three dimensions and which can automatically aspirate liquid from a cavity and dispense liquid into other cavities. The automated movement of liquid between different cavities with a pipette can be referred to as a pipetting procedure. The liquid handling system can also comprise one or more dispensing cannulas which can be part of a dispenser module placed on a work surface of the laboratory system or which can be moved in three dimensions and which can automatically dispense liquid from a reservoir into a cavity. The term “dispense” is to be understood as dispensing liquid, e.g. a reagent or a diluent, from a reservoir which is fluidically connected to the dispensing tip or as dispensing liquid from a pipetting tip into a cavity after aspiration from another cavity,

[0004] For automated liquid handling systems, process safety is increasingly important. Therefore, new air pipettes typically measure the air pressure during liquid transfer. The classification of such signals is typically not easy because there are multiple influencing parameters on the pipetting procedure, such as liquid parameters, dynamic parameters, system and pipette tip parameters, environmental parameters, etc.

[0005] There are several approaches which can evaluate the measurement data from the pressure sensor.

[0006] As an example, an ideal pressure curve can be determined based on many correct pipetting procedures. The pipetting procedure can be evaluated by comparing the measured pressure curve with the ideal pressure curve, which can only deviate from the ideal pressure curve by a predetermined percentage for a correct pipetting procedure. However, an ideal pressure curve can have to be established for every combination of volume, sample, tip type.

[0007] As a further example, a theoretical pressure curve can be determined based on pipetting, sample and environmental parameters. After the pipetting procedure, the theoretical pressure curve can be fitted to the measured pipetting curve. The fitting parameters can then be evaluated. Based on the evaluation, the liquid transfer, i.e. the pipetting procedure, can be rated as correct or incorrect. However, in this case, a theoretical model can have to be determined for the entire liquid handling system. Furthermore, the model can have to be adjusted and verified for each tip type, sample type and liquid handling system combination.

[0008] EP 1 745 851 A1 describes a pipetting device suitable for liquid classification, which is based on comparing a simulated curve and a measured curve.

[0009] In WO 2012 068 610 A1, the density of a fluid is inferred by using the measured pressure as input to a trained neural network.

[0010] US2004 / 034 479 A1 and EP 1 391 734 A2 relate to a sample dispensing device and an automatic analyzer, which uses a neuron network to analyze the waveform of pressure fluctuations.

[0011] In the article of Unver et al., "A fuzzy quality control-decision support system for improving operational reliability of liquid transfer operations in laboratory automation", Expert Systems with Applications, vo. 36, no. 4, 14 November 2018, pages 8064-8070, fuzzy logic is used to perform quality control in liquid transfer operations. SUMMARY

[0012] It is an object of the present invention to simplify the configuration and / or control of a laboratory automation system.

[0013] This object is achieved by the method for classifying a liquid handling procedure of the present invention. Further exemplary embodiments are apparent from the following description.

[0014] A first aspect of the present invention relates to a method for classifying a liquid handling procedure. The method can be performed, for example, automatically by a control device of a laboratory automation system. The liquid handling procedure can be a pipetting procedure, a dispensing procedure and / or a part of a pipetting procedure. A process performed by a laboratory automation device with a pipette automatically moving liquid between different cavities can be referred to as a pipetting procedure. An automatic dispensing of liquid by a laboratory automation device with a pipette and / or a dispensing cannula can be referred to as a dispensing procedure. The dispensing procedure can be a part of a pipetting procedure.

[0015] According to embodiments of the present invention, the method comprises receiving measurement data encoding a curve of measurement values over time during the liquid handling procedure. The measurement data can comprise liquid handling related sensor data, such as pressure measurements and / or flow measurements, e.g. of liquid and / or gas.

[0016] Measurements related to the filling level of a pipette tip and / or a container are also possible. For example, the volume aspirated can be measured by capacitive sensing within the pipette tip. The volume change in a source cavity or a destination cavity can be measured by sensing the liquid level before and after aspiration or dispensing, and the volume difference can be calculated by using the known cross section of the cavity.

[0017] The measurement data can be acquired by sensors of the laboratory automation system. For example, the pressure and / or flow rate in a gas and / or liquid in a line of a pipette of the laboratory automation system can be measured. The measurements can be performed in a volume connected to a pipette and / or a dispensing cannula, wherein the pressure is changed to perform the liquid handling procedure.

[0018] A pump can exert a pressure that is exerted on the line, and with this exerted pressure, the pipette tip can be controlled to aspirate and dispense liquid. The exerted pressure and the flowing liquid in the pipette can lead to different pressures in the line and / or different flow rates inside the line.

[0019] According to embodiments of the present invention, the method further comprises inputting the measurement data into a (artificial) neural network; and calculating at least one quality value for the liquid handling procedure by the neural network.

[0020] The neural network can be provided as a software module and / or a software library, which is complemented with a parameterization that has been generated from identified training data. The training data can be a collection of measurement data, and optionally liquid handling data of a plurality of different liquid handling procedures that have been identified. The liquid handling data can encode configurations and / or settings of the laboratory automation system. The training data can comprise one or more quality values for the corresponding liquid handling procedure.

[0021] The neural network can be a deep neural network, i.e. can comprise multiple layers, such as convolutional layers and dense layers. The neural network can comprise at least two convolutional layers and / or at least two dense layers, which are connected in a row. Different types of layers mentioned herein will be defined in the following.

[0022] A layer can be a collection of neurons, which have inputs, e.g. for receiving input values from a previous layer, and outputs, e.g. for sending output values to a next layer. A neuron can have weights and a function, which calculates an output value from input values according to the weights. An object of an object-oriented programming language can be provided to a layer. The number of neurons, the weights and the function can be provided as parameter data, e.g. can be encoded into the object. The parameter data can be seen as a parameterization of the neural network, while the layers and their interconnections can be seen as a structure of the neural network.

[0023] Measurement data from actual measurements in a laboratory automation system can be input into a neural network, which has been trained to identify the measurement data. Such an identification can be a classification, such as a classification into a successful and an erroneous liquid handling procedure. The identification can also be one or more values of a physical quantity, which is present in the corresponding liquid handling procedure, such as an aspiration volume and / or a dispense volume. Further possible physical quantities can be physical properties of the sample liquid pipetted and / or dispensed, such as specific density, viscosity, surface tension and / or wettability of the pipette tip (and / or dispense tip) surface by the sample.

[0024] Generally, a neural network based machine learning algorithm can be used to evaluate measurement data in order to identify and / or classify a liquid transfer in a laboratory automation system. A trained neural network can be used as an error detection assistant.

[0025] The neural network can even be adjusted during and / or after operation of the laboratory automation system. The error detection rate can be adjusted with respect to a positive rate of errors, identified and / or classified measurement curves can be added to the training set. A customer using the laboratory automation system can identify measurement curves and can improve the prediction capability of the neural network.

[0026] According to an embodiment of the present invention, the method further comprises controlling the laboratory automation system with the quality value. The method can be performed by a control device of the laboratory automation system during operation of the laboratory automation system. The quality value can classify a liquid handling procedure as correct or erroneous, can indicate that an aspiration volume is too small, etc. In this case, a sample and / or a sample handled by the laboratory automation system using the liquid handling procedure can be marked as erroneous and / or can be discarded when the liquid handling procedure is assumed to be incorrect.

[0027] According to embodiments of the application, the method further comprises flagging the liquid handling procedure as erroneous when the quality value indicates a failed liquid handling procedure; and / or discarding the result of the liquid handling procedure when the quality value indicates a failed liquid handling procedure.

[0028] According to embodiments of the application, the method further comprises repeating the sample and / or specimen processing with the liquid handling procedure when the quality value indicates a failed liquid handling procedure.

[0029] According to embodiments of the application, the measurement data comprises a vector of measurement values ordered by time. The pressure, the flow rate and / or more generally liquid handling sensor data can be measured over time, i.e. a measurement curve can be determined. The measurement data can comprise discrete measurement values of the curve over time. The measurement values can be concatenated into a vector ordered by time. Such a vector can be seen as a one-dimensional digitalized image of a continuous physical quantity and can thus be particularly suitable to be input into a neural network.

[0030] According to embodiments of the application, the method further comprises inputting the liquid handling data into the neural network, wherein the liquid handling data encodes a configuration and / or a setting of the laboratory automation system performing the liquid handling procedure. In addition to the measurement data, i.e. the data encoding features of the laboratory automation system, such as the pipette tip actually used, can be input into the neural network. In other words, the neural network can have been trained for different configurations and / or settings of the laboratory automation system. In this way, the same neural network can be used in different application scenarios.

[0031] The liquid handling data can comprise information about the liquid, such as the density, the viscosity and / or the type of the liquid. The liquid handling data can comprise information about the pipette and / or the disposable pipette tip, such as its size, its type, its maximum volume. The liquid handling data can comprise information about the liquid handling procedure and / or the control parameters, such as the aspiration and / or dispensing speed, the amount of liquid to be aspirated and / or to be dispensed, the length of the aspiration and / or the dispensing, the control parameters of the pump, etc.

[0032] According to embodiments of the application, the measurement data and the liquid handling data are concatenated into one vector before inputting the measurement data and the liquid handling data into the neural network. The measurement data and the liquid handling data can be seen as one type of input information and / or can be input into one input layer of the neural network.

[0033] According to embodiments of the application, the measurement data and the liquid handling data are input into different input layers of the neural network. It is also possible that both types of data are input into different input layers, which can even be connected to other different layers of the neural network. In this way, the two different sets of data can be treated differently by differently configured layers before they are treated by the same layers.

[0034] According to embodiments of the application, the neural network comprises a measurement data branch consisting of at least one layer and a liquid handling data branch consisting of at least one layer. The measurement data can be input into an input layer of the measurement data branch and the liquid handling data can be input into an input layer of the liquid handling data branch.

[0035] The measurement data branch can comprise at least two convolutional layers. However, the measurement data branch can be only one input layer. The liquid handling data branch can be only one input layer, but can also comprise one, two or more dense layers.

[0036] According to embodiments of the application, the neural network comprises a dense layer branch. For example, the output of the measurement data branch and the output of the liquid handling data branch are input into the dense layer branch. The dense layer branch can comprise at least two dense layers. The dense layer branch can comprise a probability layer at its end, which generates probability values for different classification types.

[0037] Generally, a plurality of quality values can be output by the neural network, in particular the dense layer branch. Such quality values can comprise classification values, which can be percentage values indicating probabilities for a specific classification type. Such quality values can also be estimation values, which estimate a specific physical quantity that can occur during the liquid handling procedure.

[0038] According to embodiments of the application, the neural network outputs a classification value to classify the liquid handling procedure. For example, the classification value can indicate at least one of the following: correct procedure, clot, air aspiration, too little sample, air bubble, foam, tip clogging, leak, etc. All or some of the types of problems that can occur in the liquid handling procedure can be classified. It is also possible that the classification is limited to simply indicating whether the procedure course was performed correctly or not.

[0039] According to embodiments of the application, the neural network outputs an estimation value, which estimates a physical quantity of the liquid handling procedure. For example, the estimation value estimates at least one of a dispense volume and an aspiration volume. The neural network can be trained to a model of the laboratory automation system, which outputs can be determined from the measurement data. It is not necessary to determine a model based on a mathematical function encoding the physical relationship.

[0040] According to embodiments of the present invention, the liquid handling procedure comprises lowering a pipette into a cavity containing a liquid. The pressure, flow rate and / or liquid handling sensor data can already be measured during the process of lowering the pipette into the liquid, i.e. before the pump starts working.

[0041] According to embodiments of the present invention, the liquid handling procedure comprises: aspirating liquid into the pipette by lowering the pressure in the pipette, and / or dispensing liquid of the pipette by raising the pressure in the pipette. The pressure can be lowered and / or raised by a pump connected to the pipette. The measurement data can comprise measured values during aspirating liquid into the pipette, and / or the measurement data can comprise measured values during dispensing liquid from the pipette. It is noted that the qualification of the neural network can be performed only for the aspiration, only for the dispensing and / or for both.

[0042] A further aspect of the present invention relates to a computer program for classifying a liquid handling procedure, which, when being executed by a processor, is adapted to perform the steps of the method described in the foregoing and hereinafter. The computer program can be executed in a computing device communicatively interconnected with a laboratory automation system, such as a controller of the laboratory automation system and / or such as a PC. The method can also be executed by an embedded microcontroller.

[0043] A further aspect of the present invention relates to a computer readable medium, in which such a computer program is stored. The computer readable medium can be a floppy disk, a hard disk, a USB (Universal Serial Bus) storage device, a RAM (Random Access Memory), a ROM (Read-Only Memory), an EPROM (Erasable Programmable Read-Only Memory), or a flash memory. The computer readable medium can also be a data communication network, e.g. the Internet, allowing the download of a program code. Generally, the computer readable medium can be a non-transitory or transitory medium.

[0044] A further aspect of the present invention relates to a laboratory automation system.

[0045] According to embodiments of the present invention, the laboratory automation system comprises: a liquid handling arm to carry a pipette and / or a dispensing cannula; a pump to change the pressure in a volume connected with the pipette to aspirate and dispense liquid in the pipette; a sensor device to perform measurements in the volume connected with the pipette; and a control device to control the pump and to receive measurement data from the sensor device.

[0046] The measurements can be any measurements related to the liquid handling procedure, such as pressure, flow, volume change measurements, etc. The measurement data can be or can comprise liquid handling sensor data.

[0047] Further, the control device can be adapted to perform the method described in the foregoing and hereinafter. The control device can store the neural network described in the foregoing and hereinafter.

[0048] It should be understood that features of the methods described above and below can be features of the control device, computer program and computer readable medium described above and below and vice versa.

[0049] These and other aspects of the application will become apparent from and be elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0050] Embodiments of the application are described below in more detail with reference to the drawings.

[0051] Figure 1 A laboratory automation system according to an embodiment of the application is schematically shown.

[0052] Figure 2 A flow chart of a method for classifying a liquid handling procedure according to an embodiment of the application is shown.

[0053] Figure 3 A measurement curve during a pipetting procedure is shown.

[0054] Figure 4 A plurality of measurement curves during a pipetting procedure with a clot is shown.

[0055] Figure 5 A plurality of measurement curves during a pipetting procedure with a bubble is shown.

[0056] Figure 6 An input vector of a neural network used in an embodiment of the application is schematically shown.

[0057] Figure 7 A layout of a neural network used in an embodiment of the application is schematically shown.

[0058] Figure 8 A further layout of a neural network used in an embodiment of the application is schematically shown.

[0059] Figure 9 A further layout of a neural network used in an embodiment of the application is schematically shown.

[0060] In the list of reference signs, the reference signs of the drawings, as well as their meanings, are listed in tabular form. Identical parts are provided with the same reference signs in the figures. DETAILED DESCRIPTION

[0061] Figure 1A laboratory automation system 10 is schematically shown, which comprises a pipette arm 12, which is automatically movable, to which a pipette 14 is attached. The pipette 14 can also comprise a disposable pipette tip 15, which can also be grasped and discarded by the pipette arm 12. As shown in Figure 1 The pipette tip 15 is lowered into a container 16, as shown in step S2. The container 16 can be a well of a multi-well plate, a test tube containing a sample, a container containing a reagent, etc.

[0062] The pipette arm 12 can move the pipette 14 and the pipette tip 15 in three dimensions and can lower the pipette tip 15 into the container 16 and can retract the pipette tip 15 therefrom. The container 16 and possibly the pipette tip 15 can contain a liquid 18, such as a sample or a reagent. The pipette 14 and its tip 15 are used to move and / or transport an amount of liquid between different containers 16.

[0063] The laboratory automation system 10 can also comprise a dispensing cannula, which can also be connected to the pipette arm 12, in which case the pipette arm 12 can also be regarded as a liquid handling arm. The dispensing cannula can be connected to a reservoir and can be used to dispense a liquid into a container 16.

[0064] Furthermore, the laboratory automation system 10 comprises a pump 20, which is connected with the pipette 14 via a hose 22. By means of the pump 20, a pressure can be applied to the hose 22 and the pipette 14, which causes the pipette 14 to aspirate or dispense the liquid 18.

[0065] A sensor device 24, which can be attached to the hose 22 and / or the pipette 14, is adapted to measure the pressure and / or the flow rate in the hose 22 and / or the pipette 14. The measurement data acquired by the sensor device 24 can be used to qualify a pipetting procedure, as described herein.

[0066] A control device 26 of the laboratory automation system 10, which can be part of or connected with the laboratory automation system 10, can control the pipette arm 12, the pump 20 and can receive the measurement data from the sensor device 24.

[0067] Figure 2 A flow chart of a method for classifying a liquid handling procedure is shown, which can be performed by the control device 26.

[0068] In step S10, measurement data 56 are generated. Typically, the sensor device 24 of the laboratory automation system 10 can measure the pressure and / or the flow rate over time and can generate a measurement curve therefrom. During the measurement, measurement values can be acquired over time and transmitted to the control device 26.

[0069] Figure 3The measurement curve 28, in particular the pressure curve 28, during the pipetting procedure is shown. It should be noted that everything discussed below with respect to the pressure curve and the pressure measurement also applies to the flow rate curve and the flow rate measurement. Moreover, everything discussed below can be related to a dispensing procedure performed using a dispensing cannula, where applicable.

[0070] The generation of the measurement data 56 in step S10 can be performed in parallel to the control of the laboratory automation system 10, which can also be performed by the control device 26.

[0071] Generally, the liquid 18 is transported between the two cavities 16 by sucking the liquid 18 into the pipette 14 by lowering the pressure in the pipette 14. This can be done by suitably controlling the pump 20. Afterwards, the liquid 18 in the pipette 14 is dispensed by raising the pressure in the pipette 14, which can also be performed by suitably controlling the pump 20. Before, between and after the sucking and the dispensing, the pipette 14 can be moved by the pipette arm 12 to the first cavity 16 and the second cavity 16.

[0072] Figure 3 The pressure curve 28 in the pipette 14 shows the measured pressure from grabbing the pipette tip 15 to finally letting the pipette tip 14 drop, where the time elapses from left to right. In particular, the sucking curve 30 and the dispensing curve 32 are shown enlarged.

[0073] First, the pipette tip 15 is grabbed (34) and a movement of the pipette arm 12 to the first cavity 16 is performed (36). At (38), the sucking is started. It can be seen that the pressure is lowered, which causes a negative pressure in the pipette tip 15 to suck the liquid 18. At (40), the sucking is ended and the pressure comes back to the average value. Afterwards, a further movement of the pipette arm 12 to the second cavity 16 is performed (42). At (44), the dispensing of the liquid 18 is started and at (46), the dispensing is ended. It can be seen that the pressure is increased here, so that an overpressure in the pipette tip 15 dispenses the liquid 18. Finally, a movement of the pipette arm 12 to a waste container is performed (48) and the pipette tip 15 is discarded (50) in the waste container.

[0074] During a correct sucking (30) and dispensing (32), the pressure curve looks like Figure 3 However, an error during the sucking and / or dispensing (32) causes a differently shaped pressure curve.

[0075] As an example, Figure 4 and Figure 5 a sucking curve 30 is shown, where the sucking process is not performed correctly. In particular, Figure 4 a plurality of sucking curves 30 during a pipetting procedure with clots is shown. It can be seen that all of these curves deviate in some way from the correct sucking curve 30 asFigure 3 the optimal curve shown in Fig. 6. Figure 5 The plurality of aspiration curves 30 during the pipetting procedure with air bubbles is shown. Again, it can be seen that the curves deviate in the same way from the optimal curve as shown in Fig. 6. Figure 3 the optimal curve shown in Fig. 6.

[0076] Referring back to Fig. 1, Figure 2 In step S12, measurement data 56 encoding the measurement curves 28, 30 and / or 32 measured during the pipetting procedure over time is received in the control device 26.

[0077] The control device 26 can generate a data structure in the form of a vector from the measurement values, wherein the measurement values are ordered by time. Furthermore, the data vector can be supplemented with further data, such as configuration parameters and / or parameter settings of the laboratory automation system 10.

[0078] Figure 6 A data vector 52 that can be generated by the control device 26 is shown schematically. The data vector 52 is composed of liquid handling data 54 and measurement data 56.

[0079] The liquid handling data 54 can be composed of configuration parameters and / or settings 55, for example, which can depend on the actual type of execution of the liquid handling procedure, the type of pipette tip 15 used, the type of liquid 18, etc. The control device 26 can gather such data and can put it into the data vector 52.

[0080] It is noted that the entries and / or values 55 in the liquid handling data 54 can have different sizes and / or different formats.

[0081] The measurement data 56 can be composed of the measurement values 53. The measurement data 56 can be pre-processed by the control device 26, for example, to fit a data vector 52 of a specific length.

[0082] The measurement data 56 can be arranged into a vector 52 of measurement values 53 ordered by time, i.e. the higher the index of the measurement value 53, the longer the time at which it was taken.

[0083] Referring back to Fig. 1, Figure 2 In step S14, the measurement data 56 and optionally the liquid handling data 54 are input into the neural network and at least one quality value 74, 86 for the liquid handling procedure is computed by the neural network. Reference will be made to Figures 7 to 9 Examples of neural networks will be described.

[0084] For example, Figure 7 The layout and / or structure of a neural network 57 used in embodiments of the present application is shown schematically. In Figure 7In the present case, the layout is a cascade of layers 58 to 72, each consisting of a set of neurons.

[0085] It is noted that, in addition to its layout, the neural network 57 also comprises a configuration / parameterization for its layers, such as the number of neurons and / or the number of inputs and / or the number of outputs per neuron. Furthermore, each layer also comprises weights for its inputs and a function for its outputs, which calculates the corresponding output values based on the weights.

[0086] The weights can be determined during training of the neural network 57, which is provided with a large set of training data that has been identified, such as Figures 3 to 5 The curve shown in

[0087] In the example of Figure 7 , the measurement data 56 and the liquid treatment data 54 can be concatenated into one vector 52 before being input to the neural network 57, for example as shown in Figure 6 However, it is also possible to input only a vector of measurement data 56 in the neural network 57. This can be the case when the neural network is trained only for one configuration and / or type of liquid treatment process.

[0088] Figure 7 The neural network 57 of

[0089] The input layer 58 receives a vector of input data (such as measurement data 56 and / or liquid treatment data 54) and / or pre-processes the input data in a first step.

[0090] The reshaping layer 60 changes the dimensionality of the output data with respect to the input data. The reshaping layer 60 can not be necessary if the correct format is provided directly.

[0091] The convolutional layer 62 creates feature maps by applying one or more filters (also called kernels) to a local receptive field of the input data. In the present case, the output of the reshaping layer and / or the local receptive field can be one-dimensional.

[0092] The pooling layer 64 is a layer that condenses information from the previous layer, for example by taking the maximum value of a region.

[0093] The dropout layer 66 removes some feedback information during training to generalize learning. One, some or all of the dropout layers can be optional to improve the generality of the prediction.

[0094] The flattening layer 68 converts multi-dimensional data into one-dimensional data.

[0095] The dense layer 70 is a layer that has a full one-to-one connection with the preceding layer.

[0096] The probability layer is a layer that can determine a probability value of a classifier from non-normalized input. The probability layer can be a dense layer with a softmax activation function to normalize the output, so that the sum thereof is 1.

[0097] Finally, the neural network 57 outputs one or more classification values 74, which are probability values for classifying the liquid handling procedure. For example, the classification values 74 can indicate correct procedure, clot, air aspiration, insufficient sample, air bubble, foam, tip clogging, leak.

[0098] Figure 8 A further layout of the neural network 57 that can be used in embodiments of the present application is schematically shown. In Figure 8 , the measurement data 56 and the liquid handling data 54 are input into different input layers 58, 82 of the neural network 57. Furthermore, the neural network 57 comprises a measurement data branch 76 and a liquid handling data branch 78.

[0099] The measurement data 56 is input into the input layer 58 of the measurement data branch 76, and the liquid handling data 54 is input into the input layer 82 of the liquid handling data branch 78.

[0100] The measurement data branch 76 can comprise at least two pairs of convolutional layers 62 and pooling layers 64. Furthermore, the measurement data branch 76 can comprise a reshaping layer 60 after the input layer 58 and a flattening layer 68 at the end.

[0101] The liquid handling data branch 78 consists only of the input layer 82, into which the liquid handling data 54 is input.

[0102] The outputs of the measurement data branch 76 and the liquid handling data branch 78 are input into a concatenation layer 84, which concatenates the outputs, such as two vectors, together along a specific dimension.

[0103] The output of the concatenation layer 84 is input into a dense layer branch 80 of the neural network 57. The dense layer branch 80 can comprise at least two dense layers 70. A dropout layer 66 can be arranged between the dense layers 70. The output of the dense layer branch 80 is input into a probability layer 72, such as Figure 7 the probability layer in.

[0104] It can be assumed that Figure 7 The neural network 57 of still has the measurement data branch 76 or more general convolutional branches and dense layer branches, which are connected in a row.

[0105] Figure 9 A further layout of a neural network 57 that can be used in embodiments of the application is schematically shown.

[0106] As with the neural network of Figure 8 the neural network 57 of Figure 9 comprises a dense layer branch 80 into which the output of the measurement data branch 76 and the output of the liquid handling data branch 78 are input. However, the measurement data branch 76 and the liquid handling data branch 78 consist only of one input layer 58, 82.

[0107] As with the neural network of Figure 7 and 8 the neural network 57 of Figure 9 is trained to output one or more estimation values 86 to estimate a physical quantity of the liquid handling procedure. For example, the estimation values 86 can be the dispensed volume or the aspirated volume. In this case, the training data must be provided with corresponding estimation values.

[0108] Figure 7 As with the neural network of Figure 8 the neural network 57 of Figure 9 may output estimation values 86, Figure 9 as does the neural network 57 of Figure 7 and Figure 8 the neural network 57 of outputs probability values 74, as does the neural network of

[0109] Referring back to Figure 2 in step S16, the sample and / or specimen that has been processed by the liquid handling procedure can be labeled with one or more quality values 74, 86 determined by the neural network 57.

[0110] It is also possible to discard the sample and / or specimen that has been processed by the liquid handling procedure that has been executed incorrectly as indicated by the quality values 74, 86, and / or to process the sample and / or specimen a second time.

[0111] Although the application has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the application is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art and practicing the claimed application, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or controller or other unit can fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.

[0112] List of reference signs

[0113] 10 laboratory automation system

[0114] 12 pipette arm

[0115] 14 pipette

[0116] 15 pipette tip

[0117] 16 container

[0118] 18 liquid

[0119] 20 pump

[0120] 22 hose

[0121] 24 sensor device

[0122] 26 control device

[0123] 28 pressure curve

[0124] 30 suction curve

[0125] 32 dispensing curve

[0126] 34 grasping pipette tip

[0127] 36 pipette arm movement

[0128] 38 start of suction

[0129] 40 end of suction

[0130] 42 pipette arm movement

[0131] 44 start of dispensing

[0132] 46 end of dispensing

[0133] 48 pipette arm movement

[0134] 50 discard pipette tip

[0135] 52 data vector

[0136] 53 measurement value

[0137] 54 liquid handling data

[0138] 55 configuration parameter

[0139] 56 measurement data

[0140] 57 neural network

[0141] 58 input layer

[0142] 60 reshaping layer

[0143] 62 convolution layer

[0144] 64 pooling layer

[0145] 66 dropout layer

[0146] 68 flattening layer

[0147] 70 dense layer

[0148] 72 probability determination layer

[0149] 74 quality value, classification value

[0150] 76 measurement data branch

[0151] 78 liquid handling data branch

[0152] 80 dense layer branch

[0153] 82 second input layer

[0154] 84 concatenation layer

[0155] 86 quality value, estimate value

Claims

1. A method for classifying a liquid handling procedure, the method comprising: receiving measurement data (56) encoding a measurement curve measuring over time during a liquid handling procedure; inputting the measurement data (56) into a neural network (57); inputting liquid handling data (54) into the neural network (57), wherein the liquid handling data (54) encodes a configuration and / or a setting of a laboratory automation system (10) performing the liquid handling procedure; computing at least one quality value for the liquid handling procedure with the neural network (57); wherein the neural network (57) comprises at least one measurement data branch (76) consisting of at least two convolutional layers and a liquid handling data branch (78) consisting of at least one layer; wherein the measurement data (56) is input into an input layer (58) of the measurement data branch (76) and the liquid handling data (54) is input into an input layer (82) of the liquid handling data branch (78), wherein the neural network (57) comprises a dense layer branch (80) comprising at least two dense layers, an output of the measurement data branch (76) and an output of the liquid handling data branch (78) are input into the dense layer branch (80).

2. The method according to claim 1, wherein the measurement data (56) comprises a vector (52) of measurement values ordered by time.

3. The method according to claim 1 or 2, wherein, the measurement data branch (76) comprises at least two convolutional layers (62).

4. The method according to claim 1 or 2, wherein the dense layer branch (80) comprises at least two dense layers (70).

5. The method according to claim 1 or 2, wherein, the neural network (57) outputs a classification value to classify the liquid handling procedure.

6. The method according to claim 5, wherein the classification value is indicative of at least one of: correct procedure, clot, air aspiration, sample too little, air bubble, foam, tip clogging, leak.

7. The method according to claim 1 or 2, wherein the neural network (57) outputs an estimation value estimating a physical quantity of the liquid handling procedure.

8. The method according to claim 7, wherein, the estimation value estimates at least one of: dispensed volume, aspirated volume.

9. The method according to claim 1 or 2, wherein the liquid handling procedure comprises at least one of: aspirating a liquid (18) into a pipette (14) by lowering a pressure in the pipette (14); dispensing a liquid (18) in the pipette (14) and / or a dispensing cannula by raising a pressure in the pipette (14) and / or the dispensing cannula.

10. A computer readable medium having stored therein a computer program for classifying a liquid handling procedure, which computer program, when executed by a processor, is adapted for performing the steps of the method according to one of the preceding claims.

11. A laboratory automation system (10), the system comprising: a liquid handling arm (12) for carrying a pipette (14) and / or a dispensing cannula; a pump (20) for changing the pressure in a volume (22) connected with the pipette (14) to aspirate and dispense liquid (18) in the pipette (14); a sensor device (24) for performing measurements in the volume (22) connected with the pipette (14); a control device (26) for controlling the pump (20) and receiving measurement data (56) from the sensor device (24); wherein the control device (26) is adapted to perform the method according to one of claims 1 to 9.

Images