Ultra-high spatiotemporal resolution cellular traction force quantitative method, device, equipment and medium

Abstract

The present disclosure provides a method, device, equipment and medium for quantifying cell traction force with ultra-high spatiotemporal resolution. The method for quantifying cell traction force with ultra-high spatiotemporal resolution includes: constructing and training a first neural network, a second neural network, a third neural network and a fourth neural network respectively; the fluorescence images of the fluorescent markers of the cells to be tested before and after digestion are respectively predicted by the trained first neural network, the second neural network and the third neural network at each sampling point, and integrated; the integrated displacement field is input into the fourth neural network for prediction to obtain the distribution of the traction force field of the cells to be tested. The present disclosure effectively improves the spatial resolution of the quantification of cell traction force by introducing machine learning, obtains more accurate quantitative results, reduces the computational complexity of the quantification of cell traction force, and significantly improves the temporal resolution of the quantification of cell traction force, which can be used for real-time quantitative measurement of cell traction force.

Description

Technical Field

[0001] The present disclosure relates to the technical field of cell traction force measurement, and in particular to a method, device, equipment and medium for quantifying cell traction force with ultra-high spatiotemporal resolution based on machine learning. Background Art

[0002] The mechanical interaction between cells and the extracellular matrix is ​​closely related to a series of physiological and pathological activities of cells, such as cell proliferation, differentiation, adhesion, migration, etc. Quantification of cell traction force with high temporal and spatial resolution is of great significance in applications such as cell screening and quantitative drug dosage.

[0003] So far, the research on cell traction has made a series of progress in basic theory, experimental phenomena and characterization technology. The representative traction quantitative methods can be summarized into three types: micro-pillar array method, molecular sensor method and traction force microscopy method. Among them, the micro-pillar array method is to culture cells on the substrate surface with a micro-pillar array structure, and infer the distribution of cell traction force by the deformation of the micro-pillar. This method is simple to calculate and has high measurement efficiency, but it cannot obtain the normal component of cell traction force. The molecular sensor method is realized by fluorescence resonance energy transfer, which has ultra-high resolution and can characterize the force of a single integrin in living cells, but it cannot obtain the direction information of the force. The traction force microscopy method is to culture cells on a continuous hydrogel / rubber substrate surface containing fluorescent markers, and the cell adhesion is not restricted. Based on the fluorescence images of markers before and after cell digestion obtained by fluorescence microscopy, the traction force microscopy method can simultaneously obtain the tangential component and normal component of cell traction force, and can be extended to three-dimensional situations. Therefore, the traction force microscopy method has a broader application prospect.

[0004] However, the current traction force microscopy method requires complex mathematical-mechanical theories, which has hindered its further promotion and application, and also makes it difficult to further improve the temporal-spatial resolution of the traction force microscopy method. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In response to the current technical problems, the present invention proposes a method, device, equipment and medium for quantifying cell traction force with ultra-high temporal and spatial resolution based on machine learning.

[0007] (II) Technical solution

[0008] In order to solve the above technical problems, the technical solutions adopted by the present disclosure are as follows:

[0009] The present disclosure provides a method for quantifying cell traction force with ultra-high spatiotemporal resolution, comprising: constructing and training a first neural network N1, a second neural network N2, a third neural network N3 and a fourth neural network N4 respectively; predicting the displacement of each sampling point of the fluorescence image of the fluorescent marker of the test cell before digestion and the fluorescence image of the fluorescent marker of the test cell after digestion by the trained first neural network N1, the second neural network N2 and the third neural network N3 respectively, and integrating the displacement results predicted by the first neural network N1, the second neural network N2 and the third neural network N3; and inputting the integrated displacement field into the fourth neural network N4 for prediction to obtain the distribution of the traction force field of the test cell.

[0010] In the above scheme, in the steps of respectively constructing and training the first neural network N1, the second neural network N2 and the fourth neural network N4, in the steps of respectively constructing and training the first neural network N1, the second neural network N2, the third neural network N3 and the fourth neural network N4, the first neural network N1 is a fully convolutional neural network, whose input is the fluorescent image of the fluorescent marker of the test cell before digestion and the fluorescent image of the fluorescent marker of the test cell after digestion, and the output is the corresponding cell-induced deformation field (U1, V1) composed of the displacement of all sampling points.

[0011] In the above scheme, the first neural network N1 is trained in the form of supervised learning. In order to obtain sufficient training data, the training set of the first neural network N1 is generated by numerical simulation;

[0012] First, a Gaussian function is used to approximate the fluorescence intensity distribution when imaging the fluorescent marker:

[0013] q(X, Y) = exp[-((X-X0) 2 +(Y-Y0) 2 ) / c 2 ] Formula (1)

[0014] Among them, (X0, Y0) is the center point of the current fluorescent marker, and c is the coefficient reflecting the degree of attenuation of the fluorescence intensity of the fluorescent marker;

[0015] Then, the simulated fluorescence image of the fluorescent marker after digestion of the cells to be tested is generated by the following formula:

[0016]

[0017] Where b0 is the background intensity of the image, and N is the number of fluorescent markers;

[0018] Then, the displacement fields in the specified training set all conform to the following quadratic function:

[0019]

[0020] Among them, a i is a randomly assigned coefficient, i = 1, 2, ..., 12; U and V are the displacement components along the X and Y directions, respectively, and satisfy U ≤ 3 pixels, V ≤ 3 pixels;

[0021] Therefore, the simulated fluorescence image of the fluorescent marker before cell digestion is generated by the following formula:

[0022]

[0023] in, and is the displacement of the center point of the nth fluorescent marker along the X and Y directions, that is,

[0024] Based on the above formulas (1)-(4), a series of simulated fluorescence images of the fluorescent markers of the cells to be tested before and after digestion and the corresponding displacement fields are obtained, wherein the simulated fluorescence images of the fluorescent markers of the cells to be tested before digestion and the simulated fluorescence images of the fluorescent markers of the cells to be tested after digestion are used as training data, and the corresponding displacement fields are used as training labels;

[0025] In the process of training the first neural network N1, the root mean square error is specified as the loss of network training, and the network parameters are optimized by the small batch stochastic gradient descent method; when the network training converges, the values ​​of the network parameters are recorded for subsequent calculations.

[0026] In the above scheme, in the steps of respectively constructing and training the first neural network N1, the second neural network N2, the third neural network N3 and the fourth neural network N4, the second neural network N2 is a fully convolutional neural network, whose input is the fluorescent image of the fluorescent marker before and after digestion of the cells to be tested, and the output is the deformation field caused by the corresponding cells, and the neural network is trained in the form of supervised learning.

[0027] In the above scheme, the training set of the second neural network N2 is generated by numerical simulation, and U≤1 pixel and V≤1 pixel are specified; based on the above formulas (1)-(4), a series of simulated fluorescence images of fluorescent markers before and after digestion of the cells to be tested and the corresponding displacement fields are obtained for training the second neural network N2, wherein the simulated fluorescence images of fluorescent markers before digestion of the cells to be tested and the simulated fluorescence images of fluorescent markers after digestion of the cells to be tested are used as training data, and the corresponding displacement fields are used as training labels; in the process of training the second neural network N2, the root mean square error is specified as the loss of network training, and the network parameters are optimized by the small batch stochastic gradient descent method; when the network training converges, the values ​​of the network parameters are recorded for subsequent calculations.

[0028] In the above scheme, in the steps of respectively constructing and training the first neural network N1, the second neural network N2, the third neural network N3 and the fourth neural network N4, the third neural network N3 is a fully convolutional neural network, whose input is the fluorescent image of the fluorescent marker before and after digestion of the cells to be tested, and the output is the deformation field caused by the corresponding cells, and the neural network is trained in the form of supervised learning.

[0029] In the above scheme, the training set of the third neural network N3 is generated by numerical simulation, and U≤0.5 pixel and V≤0.5 pixel are specified; based on the above formulas (1)-(4), a series of simulated fluorescence images of fluorescent markers before and after digestion of the cells to be tested and the corresponding displacement fields are obtained for training the third neural network N3, wherein the simulated fluorescence images of fluorescent markers before digestion of the cells to be tested and the simulated fluorescence images of fluorescent markers after digestion of the cells to be tested are used as training data, and the corresponding displacement fields are used as training labels; in the process of training the third neural network N3, the root mean square error is specified as the loss of network training, and the network parameters are optimized by the small batch stochastic gradient descent method; when the network training converges, the values ​​of the network parameters are recorded for subsequent calculations.

[0030] In the above scheme, in the steps of respectively constructing and training the first neural network N1, the second neural network N2, the third neural network N3 and the fourth neural network N4, the fourth neural network N4 is a fully convolutional neural network, whose input is the full-field displacement caused by the cells to be tested, and whose output is the corresponding cell traction force field.

[0031] In the above scheme, the fourth neural network N4 is trained in the form of supervised learning. In order to obtain sufficient training data, the training set of the fourth neural network N4 is generated by numerical simulation;

[0032] First, the elastic substrate to which the cells to be tested adhere is discretized into a series of uniformly distributed nodes, and the spacing between each node along the X and Y directions is 2h1 and 2h2 respectively;

[0033] Then, a rectangular grid of size 2h1×2h2 with each node as the center point is assigned a uniformly distributed force F, whose direction and magnitude are taken as random values; the corresponding displacement field is now given by:

[0034] U=GF formula (5)

[0035] Where U=[U1, V1, U2, V2,…,U M , V M ] T is the displacement vector of each node, M is the total number of nodes, U m and V mare the displacement components of the mth node along the X and Y directions respectively; F = [F x1 , F y1 , F x2 , F y2 , …, F xM , F yM ] T is the traction force vector acting in the rectangular area corresponding to each node, F xm and F ym are the traction components of the mth node along the X and Y directions respectively; G is the coefficient matrix:

[0036]

[0037] and

[0038]

[0039]

[0040]

[0041]

[0042] E is the Young's modulus of the elastic substrate to which the cell adheres; considering the inverse problem characteristics of the cell traction inversion, the fourth neural network N4 uses the classic Tikhonov regularization method to process the coefficient matrix G and obtain the regularized cell traction field f:

[0043] f=RU Formula (11)

[0044] The displacement vector U is given by equation (5), and R is the new coefficient matrix:

[0045] R=(G T G+η) -1 G T Formula (12)

[0046] Where η is the regularization coefficient; the displacement field U determined by the above formulas (5)-(12) is used as the training data of the fourth neural network N4, and the regularized cell traction force field f is used as the corresponding training label;

[0047] During the training of the fourth neural network N4, the root mean square error is specified as the loss of the fourth neural network N4 training, and the network parameters are optimized by the small batch stochastic gradient descent method; when the network training converges, the values ​​of the network parameters are recorded for subsequent calculations.

[0048] In the above scheme, the fluorescence image of the fluorescent marker of the cell to be tested before digestion and the fluorescence image of the fluorescent marker of the cell to be tested after digestion are respectively predicted by the trained first neural network N1, the second neural network N2 and the third neural network N3 for displacement of each sampling point, including:

[0049] The fluorescence images of the fluorescent markers of the cells to be tested before digestion and the fluorescence images of the fluorescent markers of the cells to be tested after digestion are input into the trained first neural network N1 for prediction. Through the mapping of the first neural network N1, a displacement field (U1, V1) composed of the displacement values ​​of all sampling points is obtained, where U1 is the displacement of the displacement field along the X direction, and V1 is the displacement of the displacement field along the Y direction;

[0050] The fluorescence images of the fluorescent markers of the cells to be tested before digestion and the fluorescence images of the fluorescent markers of the cells to be tested after digestion are input into the trained second neural network N2 for prediction. Through the mapping of the second neural network N2, a displacement field (U2, V2) composed of the displacement values ​​of all sampling points is obtained, where U2 is the displacement of the displacement field along the X direction, and V2 is the displacement of the displacement field along the Y direction;

[0051] The fluorescence images of the fluorescent markers before and after digestion of the cells to be tested are input into the trained third neural network N3 for prediction. Through the mapping of the third neural network N3, the displacement field (U3, V3) composed of the displacement values ​​of all sampling points is obtained, where U3 is the displacement of the displacement field along the X direction, and V3 is the displacement of the displacement field along the Y direction.

[0052] In the above solution, the integration of the displacement results predicted by the first neural network N1, the second neural network N2 and the third neural network N3 includes:

[0053] For the jth sampling point, if Pixel or Pixels, then take Otherwise if Pixel or Pixels, then take Otherwise take

[0054] in is the prediction result of the first neural network (N1) for the displacement of the jth sampling point along the X direction, is the prediction result of the first neural network (N1) for the displacement of the jth sampling point along the Y direction, is the prediction result of the second neural network (N2) for the displacement of the jth sampling point along the X direction, is the prediction result of the second neural network (N2) for the displacement of the jth sampling point along the Y direction, is the prediction result of the third neural network (N3) for the displacement of the jth sampling point along the X direction, is the prediction result of the third neural network (N3) for the displacement of the jth sampling point along the Y direction, U (j) is the displacement of the jth sampling point recorded along the X direction after integration, V (j) is the displacement of the jth sampling point recorded along the Y direction after integration;

[0055] The above judgment is repeatedly performed on all sampling points (J), and the integrated full-field displacement (U, V) can be obtained.

[0056] In the above scheme, the integrated displacement field (U, V) is input into the fourth neural network N4 for prediction to obtain the distribution of the traction field of the cell to be tested, including: inputting the integrated displacement field (U, V) into the fourth neural network N4 for prediction, and after mapping by the fourth neural network (N4), obtaining the distribution of the traction field of the cell to be tested and outputting it.

[0057] Another aspect of the present disclosure provides a device for quantifying cell traction force with ultra-high spatiotemporal resolution, comprising:

[0058] A neural network construction and training module, used to respectively construct and train a first neural network N1, a second neural network N2, a third neural network N3 and a fourth neural network N4;

[0059] A displacement recording module, for predicting the displacement of each sampling point using the trained first neural network N1, the second neural network N2 and the third neural network N3 respectively in the fluorescence image of the fluorescent marker of the tested cell before digestion and the fluorescence image of the fluorescent marker of the tested cell after digestion, and recording the current displacement field (U, V) composed of the displacements of all sampling points after integration; and

[0060] The cell traction force field calculation module is used to input the integrated displacement field (U, V) into the fourth neural network N4 for prediction to obtain the distribution of the cell traction force field to be measured.

[0061] In yet another aspect, the present disclosure provides an electronic device, comprising: a processor; and a memory storing a computer executable program, which, when executed by the processor, enables the processor to execute the ultra-high spatiotemporal resolution cell traction force quantification method.

[0062] In yet another aspect, the present disclosure provides a storage medium comprising computer executable instructions, which, when executed, implement the method for quantifying cell traction force with ultra-high spatiotemporal resolution.

[0063] In yet another aspect, the present disclosure provides a computer program, comprising: computer executable instructions, which are used to implement the ultra-high spatiotemporal resolution cell traction force quantification method when executed.

[0064] (III) Beneficial effects

[0065] The ultra-high spatiotemporal resolution cell traction force quantification method, device, equipment and medium provided by the present disclosure have at least one of the following beneficial effects compared with the prior art:

[0066] 1. The ultra-high spatiotemporal resolution cell traction force quantification method, device, equipment and medium provided in the present invention effectively improve the spatial resolution of cell traction force quantification by introducing machine learning, and can obtain more accurate cell traction force quantification results.

[0067] 2. The ultra-high spatiotemporal resolution cell traction force quantification method, device, equipment and medium provided in the present invention effectively reduce the computational complexity of cell traction force quantification and significantly improve the temporal resolution of cell traction force quantification by introducing machine learning, and can be used for real-time quantitative measurement of cell traction force.

[0068] 3. The ultra-high spatiotemporal resolution cell traction force quantification method, device, equipment and medium provided by the present invention significantly reduce the theoretical complexity of cell traction force quantification by introducing machine learning. It is simple and easy to use and can be used by people with non-professional backgrounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more fully understand the present disclosure and its advantages, a detailed description of the accompanying drawings is now given, in which:

[0070] Figure 1 4 is a flow chart of a method for quantifying cell traction force with ultra-high spatiotemporal resolution according to an embodiment of the present disclosure.

[0071] Figure 2 It is a specific implementation process diagram of the ultra-high spatiotemporal resolution cell traction force quantification method according to an embodiment of the present disclosure.

[0072] Figure 3A A simulated fluorescence image of a fluorescent marker before cell digestion according to an embodiment of the present disclosure is shown.

[0073] Figure 3B A simulated fluorescence image of a fluorescent marker after cell digestion according to an embodiment of the present disclosure is shown.

[0074] Figure 4A The theoretical distribution of the simulated cell traction force field according to an embodiment of the present disclosure is shown.

[0075] Figure 4B The cell traction force field calculated based on the method of the present disclosure according to an embodiment of the present disclosure is shown.

[0076] Figure 4C The figure shows the cell traction force field calculated when the sampling point spacing is 1 pixel based on the traditional traction force microscopy method according to an embodiment of the present disclosure.

[0077] Figure 4D The figure shows the cell traction force field calculated when the sampling point spacing is 5 pixels based on the traditional traction force microscopy method according to an embodiment of the present disclosure.

[0078] Figure 5A A comparison of the absolute calculation times of a conventional traction force microscopy method and the method of the present disclosure according to an embodiment of the present disclosure is shown.

[0079] Figure 5B A comparison of the relative computation times of a conventional traction force microscopy method and the method of the present disclosure according to an embodiment of the present disclosure is shown.

[0080] Figure 6 A block diagram of a device for quantifying cell traction force with ultra-high spatiotemporal resolution according to an embodiment of the present disclosure is schematically shown.

[0081] Figure 7 A block diagram of an electronic device according to an embodiment of the present disclosure is schematically shown.

[0082] [Reference Signs]

[0083] 600: Ultra-high spatiotemporal resolution cellular traction force quantification device

[0084] 601: Neural Network Construction and Training Module

[0085] 602: Displacement recording module

[0086] 603: Cellular traction field calculation module

[0087] 700: Electronic equipment

[0088] 710: Processor

[0089] 720: Memory

[0090] 721: Computer Programs DETAILED DESCRIPTION

[0091] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0092] The disclosed embodiments provide a method, device, equipment and medium for quantifying cell traction force with ultra-high temporal and spatial resolution. Figure 1 is a flow chart of a method for quantifying cell traction force with ultra-high spatiotemporal resolution according to an embodiment of the present disclosure. It should be noted that Figure 1 The examples shown are merely examples of application scenarios to which the embodiments of the present disclosure can be applied, so as to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure cannot be used in other environments or scenarios.

[0093] like Figure 1 As shown, the ultra-high spatiotemporal resolution cell traction force quantification method of the embodiment of the present disclosure comprises the following steps:

[0094] Step S1: construct and train the first neural network N1, the second neural network N2, the third neural network N3 and the fourth neural network N4 respectively.

[0095] In the disclosed embodiment, the first neural network N1 is a fully convolutional neural network, whose input is the fluorescent image of the fluorescent marker of the cell to be tested before digestion and the fluorescent image of the fluorescent marker of the cell to be tested after digestion, and the output is the corresponding cell-induced deformation field (U1, V1) composed of the displacement of all sampling points.

[0096] In the embodiment of the present disclosure, the first neural network N1 is trained in the form of supervised learning. In order to obtain sufficient training data, the training set of the first neural network N1 in the embodiment of the present disclosure is generated by numerical simulation. First, a Gaussian function is used to approximate the fluorescence intensity distribution during the imaging of the fluorescent marker:

[0097] q(X, Y) = exp[-((X-X0) 2 +(Y-Y0) 2 ) / c 2 ] Formula (1)

[0098] Wherein, (X0, Y0) is the center point of the current fluorescent marker, and c is a coefficient reflecting the attenuation degree of the fluorescence intensity of the fluorescent marker.

[0099] Then, the simulated fluorescence image of the fluorescent marker after digestion of the cells to be tested is generated by the following formula:

[0100]

[0101] Where b0 is the background intensity of the image, and N is the number of fluorescent markers;

[0102] Then, the displacement fields in the specified training set all conform to the following quadratic function:

[0103]

[0104] Among them, a i is a randomly assigned coefficient, i = 1, 2, ..., 12; U and V are the displacement components along the X direction and the Y direction, respectively, and satisfy U ≤ 3 pixels, V ≤ 3 pixels.

[0105] Therefore, the simulated fluorescence image of the fluorescent marker before cell digestion is generated by the following formula:

[0106]

[0107] in, and is the displacement of the center point of the nth fluorescent marker along the X and Y directions, that is,

[0108] Based on the above formulas (1)-(4), a series of simulated fluorescence images of fluorescent markers before and after digestion of the cells to be tested and the corresponding displacement fields are obtained, wherein the simulated fluorescence images of the fluorescent markers before digestion of the cells to be tested and the simulated fluorescence images of the fluorescent markers after digestion of the cells to be tested are used as training data, and the corresponding displacement fields are used as training labels. In the process of training the first neural network N1, the embodiment of the present disclosure specifies the root mean square error as the loss of network training, and uses a small batch stochastic gradient descent method to optimize the network parameters. When the network training converges, the values ​​of the network parameters are recorded for subsequent calculations.

[0109] In the disclosed embodiment, the second neural network N2 is a fully convolutional neural network, whose input is the fluorescent image of the fluorescent marker before and after digestion of the cells to be tested, and whose output is the deformation field caused by the corresponding cells, and the neural network is trained in the form of supervised learning.

[0110] In the embodiment of the present disclosure, similar to the first neural network N1, the training set of the second neural network N2 is also generated by numerical simulation, and U≤1 pixel and V≤1 pixel are specified. Based on the above formulas (1)-(4), a series of simulated fluorescence images of fluorescent markers before and after digestion of the cells to be tested and the corresponding displacement fields are obtained for training the second neural network N2. Among them, the simulated fluorescence images of fluorescent markers before digestion of the cells to be tested and the simulated fluorescence images of fluorescent markers after digestion of the cells to be tested are used as training data, and the corresponding displacement fields are used as training labels. In the process of training the second neural network N2, the embodiment of the present disclosure specifies the root mean square error as the loss of network training, and uses a small batch stochastic gradient descent method to optimize the network parameters. When the network training converges, the values ​​of the network parameters are recorded for subsequent calculations.

[0111] In the disclosed embodiment, the third neural network N3 is a fully convolutional neural network, whose input is the fluorescent image of the fluorescent marker before and after digestion of the cells to be tested, and whose output is the deformation field caused by the corresponding cells, and the neural network is trained in the form of supervised learning.

[0112] In the embodiment of the present disclosure, similar to the first neural network N1, the training set of the third neural network N3 is also generated by numerical simulation, and U≤0.5 pixels and V≤0.5 pixels are specified. Based on the above formulas (1)-(4), a series of simulated fluorescence images of fluorescent markers before and after digestion of the cells to be tested and the corresponding displacement fields are obtained for the training of the third neural network N3, wherein the simulated fluorescence images of fluorescent markers before digestion of the cells to be tested and the simulated fluorescence images of fluorescent markers after digestion of the cells to be tested are used as training data, and the corresponding displacement fields are used as training labels. In the process of training the third neural network N3, the embodiment of the present disclosure specifies the root mean square error as the loss of network training, and uses a small batch stochastic gradient descent method to optimize the network parameters. When the network training converges, the values ​​of the network parameters are recorded for subsequent calculations.

[0113] In the disclosed embodiment, the fourth neural network N4 is a fully convolutional neural network, whose input is the full-field displacement caused by the cells to be tested, and whose output is the corresponding cell traction force field.

[0114] In the disclosed embodiment, the fourth neural network N4 is trained in the form of supervised learning. In order to obtain sufficient training data, the training set of the fourth neural network N4 is generated by numerical simulation. First, the elastic substrate to which the cells to be tested adhere is discretized into a series of uniformly distributed nodes, and the spacing of each node along the X direction and the Y direction is 2h1 and 2h2 respectively. Subsequently, a rectangular grid with a size of 2h1×2h2 with each node as the center point is assigned a uniformly distributed force F, and its direction and magnitude are taken as random values; at this time, the corresponding displacement field is given by the following formula:

[0115] U=GF formula (5)

[0116] Where U=[U1, V1, U2, V2,…,U M , V M ] T is the displacement vector of each node, M is the total number of nodes, U m and V m are the displacement components of the mth node along the X and Y directions respectively; F = [F x1 , F y1 , F x2 , F y2 , …, F xM , F yM ] T is the traction force vector acting in the rectangular area corresponding to each node, F xm and F ym are the traction components of the mth node along the X and Y directions respectively; G is the coefficient matrix:

[0117]

[0118] and

[0119]

[0120]

[0121]

[0122]

[0123] E is the Young's modulus of the elastic substrate to which the cells adhere. Considering the inverse problem characteristics of the cell traction force inversion, the fourth neural network N4 in the embodiment of the present disclosure uses the classic Tikhonov regularization method to process the coefficient matrix G and obtain the regularized cell traction force field f:

[0124] f=RU Formula (11)

[0125] The displacement vector U is given by equation (5), and R is the new coefficient matrix:

[0126] R=(G T G+η) -1 G T Formula (12)

[0127] Where η is the regularization coefficient; the displacement field U determined by the above formulas (5)-(12) is used as the training data of the fourth neural network N4, and the regularized cell traction force field f is used as the corresponding training label. In the process of training the fourth neural network N4, the embodiment of the present disclosure specifies the root mean square error as the loss of the fourth neural network N4 training, and uses a small batch stochastic gradient descent method to optimize the network parameters; when the network training converges, the values ​​of the network parameters are recorded for subsequent calculations.

[0128] Step S2: The fluorescence images of the fluorescent markers of the cells to be tested before digestion and the fluorescence images of the fluorescent markers of the cells to be tested after digestion are respectively predicted by the trained first neural network N1, the second neural network N2 and the third neural network N3 for the displacement of each sampling point, and the displacement results predicted by the first neural network N1, the second neural network N2 and the third neural network N3 are integrated.

[0129] In the embodiment of the present disclosure, the traction force field of the cell to be tested is calculated based on the fluorescent image of the fluorescent marker of the cell to be tested before digestion and the fluorescent image of the fluorescent marker of the cell to be tested after digestion, which is based on the first neural network N1, the second neural network N2, the third neural network N3 and the fourth neural network N4 respectively constructed and trained in step S1. The specific implementation process is as follows: Figure 2 shown. The fluorescence image of the fluorescent marker of the cell to be tested before digestion and the fluorescence image of the fluorescent marker of the cell to be tested after digestion are input into the trained first neural network N1 for prediction, and through the mapping of the first neural network N1, a displacement field (U1, V1) composed of the displacement values ​​of all sampling points is obtained, wherein U1 is the displacement of the displacement field along the X direction, and V1 is the displacement of the displacement field along the Y direction; the fluorescence image of the fluorescent marker of the cell to be tested before digestion and the fluorescence image of the fluorescent marker of the cell to be tested after digestion are input into the trained second neural network N2 for prediction, and through the mapping of the second neural network N2, a displacement field (U2, V2) composed of the displacement values ​​of all sampling points is obtained, wherein U2 is the displacement of the displacement field along the X direction, and V2 is the displacement of the displacement field along the Y direction; the fluorescence image of the fluorescent marker of the cell to be tested before digestion and the fluorescence image of the fluorescent marker of the cell to be tested after digestion are input into the trained third neural network N3 for prediction, and through the mapping of the third neural network N3, a displacement field (U3, V3) composed of the displacement values ​​of all sampling points is obtained, wherein U3 is the displacement of the displacement field along the X direction, and V3 is the displacement of the displacement field along the Y direction;

[0130] The specific process of integrating the displacement results predicted by the first neural network N1, the second neural network N2 and the third neural network N3 is as follows:

[0131] For the jth sampling point, if Pixel or Pixels, then take Otherwise if Pixel or Pixels, then take Otherwise take in is the prediction result of the first neural network N1 for the displacement of the jth sampling point along the X direction, is the prediction result of the first neural network N1 for the displacement of the jth sampling point along the Y direction, is the prediction result of the second neural network N2 for the displacement of the jth sampling point along the X direction, is the prediction result of the second neural network N2 for the displacement of the jth sampling point along the Y direction, is the prediction result of the third neural network N3 for the displacement of the jth sampling point along the X direction, is the prediction result of the third neural network N3 for the displacement of the jth sampling point along the Y direction, U (j) is the displacement of the jth sampling point recorded along the X direction after integration, V (j) is the displacement of the jth sampling point recorded after integration along the Y direction. Repeat the above judgment for all J sampling points to obtain the integrated full-field displacement (U, V).

[0132] Step S3: Input the integrated displacement field (U, V) into the fourth neural network N4 for prediction to obtain the distribution of the traction force field of the cell to be tested.

[0133] In the disclosed embodiment, the integrated displacement field (U, V) is input into the fourth neural network N4 for prediction, and after mapping by the fourth neural network N4, the distribution of the traction force field of the cell to be measured is obtained and output.

[0134] Based on the above steps S1-S3, the cell traction force field distribution with high spatiotemporal resolution can be accurately and efficiently obtained. After the first to fourth neural networks (N1-N4) are constructed and trained, each time the method of the disclosed embodiment is used, only steps S2 and S3 need to be executed without iterative solution, so the complexity of the calculation is significantly reduced and the calculation efficiency is significantly improved.

[0135] Example 1

[0136] The following is a detailed description of the ultra-high spatiotemporal resolution cell traction force quantification method provided by the present disclosure in conjunction with a specific simulation example. In the current embodiment of the present disclosure, the fluorescent marker simulation images before and after cell digestion are shown in FIG. Figure 3A and Figure 3B As shown, Figure 3A The white dashed lines in the middle indicate the outlines of the simulated cells, and the white solid lines indicate the area to be calculated. Figure 3A The theoretical traction field generated by the cells is as follows Figure 4A As shown, the magnitude of the traction force is a random value ranging from 0 to 1800 Pa, the direction points to the centroid of the cell, and the Young's modulus of the elastic substrate is taken as 10 kPa.

[0137] The specific steps of the method for quantifying the traction force field of the cell to be tested with ultra-high spatiotemporal resolution based on the current embodiment of the present disclosure are as follows:

[0138] Step S1: construct and train a first neural network N1, a second neural network N2, a third neural network N3 and a fourth neural network N4 respectively;

[0139] Step S2: The fluorescence images of the fluorescent markers of the cells to be tested before digestion and the fluorescence images of the fluorescent markers of the cells to be tested after digestion are predicted by the trained first neural network N1, the second neural network N2 and the third neural network N3 for the displacement of each sampling point, and integrated; and

[0140] Step S3: Input the integrated displacement field into the fourth neural network N4 for prediction to obtain the distribution of the traction force field of the cell to be tested.

[0141] In the current embodiment of the present disclosure, the first neural network N1 constructed and trained in step S1 is a fully convolutional neural network containing 6 hidden layers, whose input is the fluorescent image of the fluorescent marker before and after cell digestion, and the output is the deformation field caused by the corresponding cells. The convolution kernel size of the network is 5×5, and the pooling window size is 3×3. The network is trained in the form of supervised learning.

[0142] In order to obtain sufficient training data, the training set of the first neural network N1 in the present disclosure is generated by numerical simulation. First, the present disclosure uses a Gaussian function to approximate the fluorescence intensity distribution during the imaging of the fluorescent marker:

[0143] q(X, Y) = exp[-((X-X0) 2 +(Y-Y0) 2 ) / c 2 ] Formula (1)

[0144] Wherein, (X0, Y0) is the center point of the current fluorescent marker, and c is a coefficient reflecting the attenuation degree of the fluorescence intensity of the fluorescent marker.

[0145] Furthermore, the simulated fluorescence image of the fluorescent marker after cell digestion can be generated by the following formula:

[0146]

[0147] Where b0 is the background intensity of the image and N is the number of fluorescent markers.

[0148] Subsequently, the present disclosure specifies that the displacement fields in the training set all conform to the following quadratic function:

[0149]

[0150] Among them, a i is a randomly assigned coefficient, i = 1, 2, ..., 12. U and V are displacement components along the X and Y directions, respectively, and satisfy U ≤ 3 pixels, V ≤ 3 pixels.

[0151] Therefore, the simulated fluorescence image of the fluorescent marker before cell digestion can be generated by the following formula

[0152]

[0153] in, and is the displacement of the center point of the nth fluorescent marker along the X and Y directions, that is,

[0154] Based on formula (1-4), a series of simulated fluorescence images of fluorescent markers before and after cell digestion and the corresponding displacement fields can be easily obtained, wherein the simulated fluorescence images of fluorescent markers before cell digestion and the simulated fluorescence images of fluorescent markers after cell digestion are used as training data, and the corresponding displacement fields are used as training labels. The present disclosure specifies the root mean square error as the loss of network training, and uses a small batch stochastic gradient descent method to optimize the network parameters, with a batch size of 64 and a learning rate of 0.01. When the network training converges, the values ​​of the network parameters are recorded for subsequent calculations.

[0155] In the current embodiment of the present disclosure, the second neural network N2 constructed and trained in step S1 is a fully convolutional neural network containing 6 hidden layers, whose input is the fluorescent image of the fluorescent marker before and after cell digestion, and the output is the deformation field caused by the corresponding cell. The convolution kernel size of the network is 5×5, and the pooling window size is 3×3. The network is trained in the form of supervised learning. Similar to the first neural network N1, the training set of the second neural network N2 in the present disclosure is also generated by numerical simulation, and U≤1 pixel and V≤1 pixel are specified. Based on formula (1-4), a series of simulated fluorescent images of fluorescent markers before and after cell digestion and the corresponding displacement fields can be easily obtained for training the second neural network N2. Among them, the simulated fluorescent images of fluorescent markers before cell digestion and the simulated fluorescent images of fluorescent markers after cell digestion are used as training data, and the corresponding displacement fields are used as training labels. The present disclosure specifies the root mean square error as the loss of network training, and uses a small batch stochastic gradient descent method to optimize the network parameters, with a batch size of 64 and a learning rate of 0.01. When the network training converges, the values ​​of the network parameters are recorded for subsequent calculations.

[0156] In the current embodiment of the present disclosure, the third neural network N3 constructed and trained in step S1 is a fully convolutional neural network containing 6 hidden layers, whose input is the fluorescent image of the fluorescent marker before and after cell digestion, and the output is the deformation field caused by the corresponding cell. The convolution kernel size of the network is 5×5, and the pooling window size is 3×3. The network is trained in the form of supervised learning. Similar to the first neural network N1, the training set of the third neural network N3 in the present disclosure is also generated by numerical simulation, and U≤0.5 pixels and V≤0.5 pixels are specified. Based on formula (1-4), a series of simulated fluorescent images of fluorescent markers before and after cell digestion and the corresponding displacement fields can be easily obtained for training the third neural network N3. Among them, the simulated fluorescent images of fluorescent markers before cell digestion and the simulated fluorescent images of fluorescent markers after cell digestion are used as training data, and the corresponding displacement fields are used as training labels. The present disclosure specifies the root mean square error as the loss of network training, and uses a small batch stochastic gradient descent method to optimize the network parameters, with a batch size of 64 and a learning rate of 0.01. When the network training converges, the values ​​of the network parameters are recorded for subsequent calculations.

[0157] In the current embodiment of the present disclosure, the fourth neural network N4 constructed and trained in step S1 is a fully convolutional neural network that does not contain a hidden layer, whose input is the full-field displacement caused by the cells, and the output is the corresponding cell traction force field. The convolution kernel size is 14×7, and the neural network is trained in the form of supervised learning. In order to obtain sufficient training data, the training set of the fourth neural network N4 in the present disclosure is generated by numerical simulation. First, the elastic substrate to which the cells adhere is discretized into a series of uniformly distributed nodes, and the spacing of each node along the X and Y directions is 2h1 and 2h2, respectively. Subsequently, a rectangular grid with a size of 2h1×2h2 with each node as the center point is assigned a uniformly distributed force F, and its direction and magnitude are taken as random values. At this time, the corresponding displacement field can be given by the following formula:

[0158] U=GF formula (5)

[0159] Where U=[U1, V1, U2, V2,…,U M , V M ] T is the displacement vector of each node, M is the total number of nodes, U m and V m are the displacement components of the mth node along the X and Y directions respectively. x1 , F y1 , F x2 , F y2 , …, F xM , F yM ] T is the traction force vector acting in the rectangular area corresponding to each node, F xm and F ym are the traction components of the mth node along the X and Y directions respectively. G is the coefficient matrix:

[0160]

[0161] and

[0162]

[0163]

[0164]

[0165]

[0166] E is the Young's modulus of the elastic substrate to which the cells adhere. Considering the inverse problem characteristics of the cell traction force inversion, the present disclosure uses the classic Tikhonov regularization method to process the coefficient matrix G and obtain the regularized cell traction force field f:

[0167] f=RU Formula (11)

[0168] The displacement vector U is given by equation (5), and R is the new coefficient matrix:

[0169] R=(G T G+η) -1 G T Formula (12)

[0170] Where η is the regularization coefficient, which is determined by the mature L-corner discovery algorithm. The displacement field U determined by the above formulas is used as the training data of the fourth neural network N4, and the regularized cell traction force field f is used as the corresponding training label. The present disclosure specifies the root mean square error as the loss of the fourth neural network N4 training, and uses a small batch stochastic gradient descent method to optimize the network parameters, with a batch size of 64 and a learning rate of 0.001. When the network training converges, the values ​​of the network parameters are recorded for subsequent calculations.

[0171] In the current embodiment of the present disclosure, the fluorescence images of the fluorescent markers of the cells to be tested before digestion and the fluorescence images of the fluorescent markers of the cells to be tested after digestion in step S2 are respectively predicted by the trained first neural network N1, the second neural network N2 and the third neural network N3 to predict the displacement of each sampling point, and the integration is based on the first to fourth neural networks (N1-N4) constructed and trained in step S1. The specific implementation process is as follows: Figure 2 shown.

[0172] The fluorescence images of the fluorescent markers of the cells to be tested before digestion and the fluorescence images of the fluorescent markers of the cells to be tested after digestion are input into the trained first neural network N1 for prediction. Through the mapping of the first neural network N1, a displacement field (U1, V1) composed of the displacement values ​​of all sampling points is obtained, where U1 is the displacement of the displacement field along the X direction, and V1 is the displacement of the displacement field along the Y direction;

[0173] The fluorescence images of the fluorescent markers of the cells to be tested before digestion and the fluorescence images of the fluorescent markers of the cells to be tested after digestion are input into the trained second neural network N2 for prediction. Through the mapping of the second neural network N2, a displacement field (U2, V2) composed of the displacement values ​​of all sampling points is obtained, where U2 is the displacement of the displacement field along the X direction, and V2 is the displacement of the displacement field along the Y direction;

[0174] The fluorescence images of the fluorescent markers of the cells to be tested before digestion and the fluorescence images of the fluorescent markers of the cells to be tested after digestion are input into the trained third neural network N3 for prediction. Through the mapping of the third neural network N3, a displacement field (U3, V3) composed of the displacement values ​​of all sampling points is obtained, where U3 is the displacement of the displacement field along the X direction, and V3 is the displacement of the displacement field along the Y direction;

[0175] The specific process of integrating the displacement results predicted by the first neural network N1, the second neural network N2 and the third neural network N3 is as follows:

[0176] For the jth sampling point, if Pixel or Pixels, then take Otherwise if Pixel or Pixels, then take Otherwise take

[0177] in is the prediction result of the first neural network N1 for the displacement of the jth sampling point along the X direction, is the prediction result of the first neural network N1 for the displacement of the jth sampling point along the Y direction, is the prediction result of the second neural network N2 for the displacement of the jth sampling point along the X direction, is the prediction result of the second neural network N2 for the displacement of the jth sampling point along the Y direction, is the prediction result of the third neural network N3 for the displacement of the jth sampling point along the X direction, is the prediction result of the third neural network N3 for the displacement of the jth sampling point along the Y direction, U (j) is the displacement of the jth sampling point recorded along the X direction after integration, V (j) is the displacement of the jth sampling point recorded along the Y direction after integration.

[0178] The above determination is repeated for all J sampling points to obtain the integrated full-field displacement (U, V).

[0179] Step S3: Input the integrated displacement field (U, V) into the fourth neural network N4 for prediction to obtain the distribution of the traction force field of the cell to be tested.

[0180] In the disclosed embodiment, the integrated displacement field (U, V) is input into the fourth neural network N4 for prediction, and after mapping by the fourth neural network N4, the distribution of the traction force field of the cell to be measured is obtained and output.

[0181] Based on the above steps S1-S3, the cell traction force field distribution with high spatiotemporal resolution can be accurately and efficiently obtained. After the first to fourth neural networks (N1-N4) are constructed and trained, each time the method of the disclosed embodiment is used, only steps S2 and S3 need to be executed without iterative solution, so the complexity of the calculation is significantly reduced and the calculation efficiency is significantly improved.

[0182] The distribution of the cell traction force field measured by the ultra-high spatiotemporal resolution cell traction force quantitative method according to the embodiment of the present disclosure is as follows: Figure 4B As shown in Figure 2, for comparison, the traditional traction force microscopy method is also used to calculate the current cell traction field. Considering the low computational efficiency of the traditional traction force microscopy method, two sampling intervals are selected, namely, sampling interval 1 pixel and sampling interval 5 pixels. The cell traction force measurement results based on the traditional cell traction force microscopy method are shown in Figure 2. Figure 4C and Figure 4D The absolute and relative values ​​of the corresponding calculation time are shown in Figure 5A and Figure 5B Given in.

[0183] By comparing the results obtained by the traditional traction force microscopy method and the method disclosed in the present invention, it can be seen that the spatial resolution of the traditional traction force microscopy method decreases with the increase of the sampling interval, and the temporal resolution increases with the increase of the sampling interval. This means that the traditional traction force microscopy method needs to sacrifice spatial resolution in exchange for temporal resolution, or sacrifice temporal resolution in exchange for spatial resolution, and it is difficult to obtain a cell traction force field with high spatiotemporal resolution. The method disclosed in the present invention can obtain quantitative results of cell traction force with ultra-high spatiotemporal resolution, and not only the measurement accuracy is higher than that of the traditional traction force microscopy method (as shown in Figure 4), but also the efficiency can be improved by three orders of magnitude compared with the traditional traction force microscopy method (as shown in Figure 5). Through this process and results, the feasibility and beneficial effects of the present invention can be seen.

[0184] From the above embodiments, it can be seen that the ultra-high spatiotemporal resolution cell traction force quantification method provided by the present disclosure, by introducing machine learning, on the one hand effectively improves the spatial resolution of cell traction force quantification, and can obtain more accurate quantitative results; on the other hand, it effectively reduces the computational complexity of cell traction force quantification, significantly improves the temporal resolution of cell traction force quantification, and can be used for real-time quantitative measurement of cell traction force. In addition, the ultra-high spatiotemporal resolution cell traction force quantification method provided by the present disclosure significantly reduces the theoretical complexity of cell traction force quantification, is simple and easy to use, and can be used by non-professional background personnel.

[0185] based on Figure 1 The ultra-high spatiotemporal resolution cell traction force quantification method according to the embodiment of the present disclosure is shown, Figure 6 A block diagram of a device for quantifying cell traction force with ultra-high spatiotemporal resolution according to an embodiment of the present disclosure is schematically shown.

[0186] like Figure 6As shown, the ultra-high spatiotemporal resolution cell traction force quantification device 600 provided by the embodiment of the present disclosure includes a neural network construction and training module 601, a displacement recording module 602 and a cell traction force field calculation module 603. Among them: the neural network construction and training module 601 is used to respectively construct and train the first neural network N1, the second neural network N2, the third neural network N3 and the fourth neural network N4; the displacement recording module 602 is used to respectively predict the displacement of each sampling point from the fluorescent image of the fluorescent marker before digestion of the cell to be tested and the fluorescent image of the fluorescent marker after digestion of the cell to be tested by the trained first neural network N1, the second neural network N2 and the third neural network N3, and after integration, record the current displacement field (U, V) composed of the displacement of all sampling points; the cell traction force field calculation module 603 is used to input the integrated displacement field (U, V) into the fourth neural network N4 for prediction, and obtain the distribution of the traction force field of the cell to be tested.

[0187] It should be understood that the neural network construction and training module 601, the displacement recording module 602 and the cell traction force field calculation module 603 can be combined into one module, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module.

[0188] According to an embodiment of the present disclosure, at least one of the neural network construction and training module 601, the displacement recording module 602, and the cell traction force field calculation module 603 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or any other reasonable way of integrating or packaging a circuit, such as hardware or firmware, or an appropriate combination of software, hardware, and firmware. Alternatively, at least one of the neural network construction and training module 601, the displacement recording module 602, and the cell traction force field calculation module 603 may be at least partially implemented as a computer program module, and when the program is run by a computer, the function of the corresponding module may be executed.

[0189] The disclosed embodiment also provides an electronic device for quantifying cell traction force with ultra-high spatiotemporal resolution, such as Figure 7 As shown, Figure 7 The electronic device 700 includes: one or more processors 710; a memory 720 storing a computer executable program, which, when executed by the processor 710, enables the processor 710 to implement Figure 1 The method for quantifying cellular traction forces with ultra-high spatiotemporal resolution is shown.

[0190] Specifically, the processor 710 may include, for example, a general-purpose microprocessor, an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 710 may also include an onboard memory for cache purposes. The processor 710 may be a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0191] The memory 720, for example, can be any medium capable of containing, storing, conveying, propagating or transmitting instructions. For example, the readable storage medium can include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device, device or propagation medium. Specific examples of readable storage media include: a magnetic storage device, such as a magnetic tape or a hard disk (HDD); an optical storage device, such as a compact disk (CD-ROM); a memory, such as a random access memory (RAM) or flash memory; and / or a wired / wireless communication link.

[0192] The memory 720 may include a computer program 721 , which may include code / computer executable instructions, which when executed by the processor 710 causes the processor 710 to perform a method according to an embodiment of the present disclosure or any variation thereof.

[0193] The computer program 721 may be configured to have, for example, a computer program code including a computer program module. For example, in an example embodiment, the code in the computer program 721 may include at least one program module, for example, including a module 721A, a module 721B, ... It should be noted that the division method and number of modules are not fixed, and those skilled in the art may use appropriate program modules or program module combinations according to actual conditions, and when these program module combinations are executed by the processor 710, the processor 710 may execute the method according to the embodiment of the present disclosure or any variation thereof.

[0194] The embodiments of the present disclosure also provide a storage medium containing computer executable instructions, which may be contained in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The storage medium containing computer executable instructions carries one or more programs, and when the one or more programs are executed, the ultra-high spatiotemporal resolution cell traction force quantification method according to the embodiments of the present disclosure is implemented.

[0195] According to an embodiment of the present disclosure, the storage medium containing computer executable instructions may be a computer readable signal medium or a computer readable storage medium or any combination of the above two. The computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, optical cable, radio frequency signal, etc., or any suitable combination of the above.

[0196] The present disclosure also provides a computer program, which includes: computer executable instructions, which are used to implement the ultra-high spatiotemporal resolution cell traction force quantification method according to the embodiment of the present disclosure when the instructions are executed.

[0197] So far, the present disclosure has been described in detail in conjunction with the accompanying drawings. Based on the above description, those skilled in the art should have a clear understanding of the present disclosure.

[0198] It should be noted that the implementation methods not shown or described in the drawings or the text of the specification are all forms known to ordinary technicians in the relevant technical field and are not described in detail. In addition, the above definitions of each element are not limited to the various specific structures, shapes or methods mentioned in the embodiments, and ordinary technicians in the field can simply change or replace them.

[0199] Of course, according to actual needs, the present disclosure may also include other parts, which are irrelevant to the innovation of the present disclosure and will not be described here.

[0200] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various disclosed aspects, in the above description of the exemplary embodiments of the present disclosure, the various features of the present disclosure are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: the claimed disclosure requires more features than the features explicitly recited in each claim. More specifically, as reflected in the claims below, the disclosed aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present disclosure.

[0201] In addition, in the drawings or descriptions, similar or identical parts use the same figure numbers. The technical features in the various embodiments exemplified in the specification can be freely combined to form new solutions without conflict. In addition, each claim can be used as an embodiment alone or the technical features in each claim can be combined as a new embodiment. Furthermore, the elements or implementations not shown or described in the drawings are forms known to ordinary technicians in the relevant technical field. In addition, although demonstrations of parameters containing specific values ​​may be provided herein, it should be understood that the parameters do not need to be exactly equal to the corresponding values, but can be approximated to the corresponding values ​​within an acceptable error tolerance or design constraint.

[0202] Unless there are technical obstacles or contradictions, the above-mentioned various embodiments of the present disclosure can be freely combined to form additional embodiments, and these additional embodiments are all within the protection scope of the present disclosure.

[0203] Although the present disclosure is described in conjunction with the accompanying drawings, the embodiments disclosed in the accompanying drawings are intended to exemplify the preferred embodiments of the present disclosure and should not be construed as limiting the present disclosure. The dimensional ratios in the accompanying drawings are merely illustrative and should not be construed as limiting the present disclosure.

[0204] Although some embodiments of the present general inventive concept have been shown and described, it will be appreciated by those skilled in the art that changes may be made to these embodiments without departing from the principles and spirit of the present general inventive concept, the scope of which is defined by the claims and their equivalents.

[0205] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present disclosure. It should be understood that the above description is only a specific embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A method for quantifying cell traction forces with ultra-high spatiotemporal resolution, characterized in that: include: Constructing and training a first neural network (N1), a second neural network (N2), a third neural network (N3) and a fourth neural network (N4) respectively; The fluorescence images of the fluorescent markers of the cells to be tested before digestion and the fluorescence images of the fluorescent markers of the cells to be tested after digestion are respectively predicted by the trained first neural network (N1), the second neural network (N2) and the third neural network (N3) for the displacement of each sampling point, and the displacement results predicted by the first neural network (N1), the second neural network (N2) and the third neural network (N3) are integrated; as well as The integrated displacement field is input into the fourth neural network (N4) for prediction to obtain the distribution of the traction force field of the cell to be tested; The fluorescence images of the fluorescent markers of the cells to be tested before digestion and the fluorescence images of the fluorescent markers of the cells to be tested after digestion are respectively predicted by the trained first neural network (N1), the second neural network (N2) and the third neural network (N3) for displacement of each sampling point, including: The fluorescence images of the fluorescent markers of the cells to be tested before digestion and the fluorescence images of the fluorescent markers of the cells to be tested after digestion are input into the trained first neural network (N1) for prediction, and a displacement field (U1, V1) composed of the displacement values ​​of all sampling points is obtained through the mapping of the first neural network (N1), wherein U1 is the displacement of the displacement field along the X direction, and V1 is the displacement of the displacement field along the Y direction; The fluorescence images of the fluorescent markers of the cells to be tested before digestion and the fluorescence images of the fluorescent markers of the cells to be tested after digestion are input into the trained second neural network (N2) for prediction. Through the mapping of the second neural network (N2), a displacement field (U2, V2) composed of the displacement values ​​of all sampling points is obtained, wherein U2 is the displacement of the displacement field along the X direction, and V2 is the displacement of the displacement field along the Y direction; The fluorescence images of the fluorescent markers before and after digestion of the cells to be tested are input into the trained third neural network (N3) for prediction. Through the mapping of the third neural network (N3), a displacement field (U3, V3) composed of the displacement values ​​of all sampling points is obtained, wherein U3 is the displacement of the displacement field along the X direction, and V3 is the displacement of the displacement field along the Y direction; The integration of the displacement results predicted by the first neural network (N1), the second neural network (N2) and the third neural network (N3) includes: For the jth sampling point, if Pixel or V1 (j) >1 pixel, then take V (j) =V1 (j) ,otherwise if Pixel or Pixels, then take otherwise Pick in is the prediction result of the first neural network (N1) for the displacement of the jth sampling point along the X direction, V1 (j) is the prediction result of the first neural network (N1) for the displacement of the jth sampling point along the Y direction, is the prediction result of the second neural network (N2) for the displacement of the jth sampling point along the X direction, is the prediction result of the second neural network (N2) for the displacement of the jth sampling point along the Y direction, is the prediction result of the third neural network (N3) for the displacement of the jth sampling point along the X direction, is the prediction result of the third neural network (N3) for the displacement of the jth sampling point along the Y direction, U (j) is the displacement of the jth sampling point recorded along the X direction after integration, V (j) is the displacement of the jth sampling point recorded along the Y direction after integration; The above judgment is repeated for all J sampling points to obtain the integrated full-field displacement (U, V).

2. The ultra-high spatiotemporal resolution cell traction force quantification method according to claim 1, characterized in that: In the steps of respectively constructing and training the first neural network (N1), the second neural network (N2), the third neural network (N3) and the fourth neural network (N4), the first neural network (N1) is a fully convolutional neural network, whose input is the fluorescent image of the fluorescent marker before digestion of the cells to be tested and the fluorescent image of the fluorescent marker after digestion of the cells to be tested, and the output is the corresponding cell-induced displacement field (U1, V1) composed of the displacement of all sampling points.

3. The ultra-high spatiotemporal resolution cell traction force quantification method according to claim 2, characterized in that: The first neural network (N1) is trained in a supervised learning form. In order to obtain sufficient training data, the training set of the first neural network (N1) is generated by numerical simulation; First, a Gaussian function is used to approximate the fluorescence intensity distribution when imaging the fluorescent marker: Among them, (X0, Y0) is the center point of the current fluorescent marker, and c is the coefficient reflecting the degree of fluorescence intensity attenuation of the fluorescent marker; Then, the simulated fluorescence image of the fluorescent marker after digestion of the cells to be tested is generated by the following formula: Where b0 is the background intensity of the image, and N is the number of fluorescent markers; Then, the displacement fields in the specified training set all conform to the following quadratic function: Among them, a i is a randomly assigned coefficient, i=1,2,...,12; U and V are the displacement components along the X and Y directions respectively, and satisfy U≤3 pixels, V≤3 pixels; Therefore, the simulated fluorescence image of the fluorescent marker before cell digestion is generated by the following formula: in, and is the displacement of the center point of the nth fluorescent marker along the X and Y directions, that is, Based on the above formulas (1)-(4), a series of simulated fluorescence images of the fluorescent markers of the cells to be tested before and after digestion and the corresponding displacement fields are obtained, wherein the simulated fluorescence images of the fluorescent markers of the cells to be tested before digestion and the simulated fluorescence images of the fluorescent markers of the cells to be tested after digestion are used as training data, and the corresponding displacement fields are used as training labels; In the process of training the first neural network (N1), the root mean square error is specified as the loss of network training, and the network parameters are optimized using the mini-batch stochastic gradient descent method; when the network training converges, the values ​​of the network parameters are recorded for subsequent calculations.

4. The ultra-high spatiotemporal resolution cell traction force quantification method according to claim 3, characterized in that: In the steps of respectively constructing and training the first neural network (N1), the second neural network (N2), the third neural network (N3) and the fourth neural network (N4), the second neural network (N2) is a fully convolutional neural network, whose input is the fluorescent image of the fluorescent marker before and after digestion of the cells to be tested, and the output is the displacement field caused by the corresponding cells, and the neural network is trained in the form of supervised learning.

5. The ultra-high spatiotemporal resolution cell traction force quantification method according to claim 4, characterized in that: The training set of the second neural network (N2) is generated by numerical simulation, and specifies U≤1 pixel and V≤1 pixel; Based on the above formulas (1)-(4), a series of simulated fluorescence images of the fluorescent markers of the cells to be tested before and after digestion and the corresponding displacement fields are obtained for training the second neural network (N2), wherein the simulated fluorescence images of the fluorescent markers of the cells to be tested before digestion and the simulated fluorescence images of the fluorescent markers of the cells to be tested after digestion are used as training data, and the corresponding displacement fields are used as training labels; In the process of training the second neural network (N2), the root mean square error is specified as the loss of network training, and the network parameters are optimized using the mini-batch stochastic gradient descent method; when the network training converges, the values ​​of the network parameters are recorded for subsequent calculations.

6. The method for quantifying cell traction force with ultra-high temporal and spatial resolution according to claim 5, characterized in that: In the steps of respectively constructing and training the first neural network (N1), the second neural network (N2), the third neural network (N3) and the fourth neural network (N4), the third neural network (N3) is a fully convolutional neural network, whose input is the fluorescent image of the fluorescent marker before and after digestion of the cells to be tested, and the output is the displacement field caused by the corresponding cells, and the neural network is trained in the form of supervised learning.

7. The ultra-high spatiotemporal resolution cell traction force quantification method according to claim 6, characterized in that: The training set of the third neural network (N3) is generated by numerical simulation, and specifies U≤0.5 pixel and V≤0.5 pixel; Based on the above formulas (1)-(4), a series of simulated fluorescence images of the fluorescent markers of the cells to be tested before and after digestion and the corresponding displacement fields are obtained for training the third neural network (N3), wherein the simulated fluorescence images of the fluorescent markers of the cells to be tested before digestion and the simulated fluorescence images of the fluorescent markers of the cells to be tested after digestion are used as training data, and the corresponding displacement fields are used as training labels; In the process of training the third neural network (N3), the root mean square error is specified as the loss of network training, and the network parameters are optimized by the mini-batch stochastic gradient descent method; when the network training converges, the values ​​of the network parameters are recorded for subsequent calculations.

8. The method for quantifying cell traction force with ultra-high temporal and spatial resolution according to claim 7, characterized in that: In the steps of respectively constructing and training the first neural network (N1), the second neural network (N2), the third neural network (N3) and the fourth neural network (N4), the fourth neural network (N4) is a fully convolutional neural network, whose input is the full-field displacement caused by the cells to be tested, and whose output is the corresponding cell traction force field.

9. The ultra-high spatiotemporal resolution cell traction force quantification method according to claim 8, characterized in that: The fourth neural network (N4) is trained in a supervised learning form. In order to obtain sufficient training data, the training set of the fourth neural network (N4) is generated by numerical simulation; First, the elastic substrate to which the cells to be tested adhere is discretized into a series of uniformly distributed nodes, and the spacing between each node along the X and Y directions is 2h1 and 2h2 respectively; Then, a rectangular grid of size 2h1×2h2 with each node as the center point is assigned a uniformly distributed force F, whose direction and magnitude are taken as random values; the corresponding displacement field is now given by: U=GF formula (5) where U=[U1,V1,U2,V2,...,U M ,V M ] T is the displacement vector of each node, M is the total number of nodes, U m and V m are the displacement components of the mth node along the X and Y directions respectively; F = [F x1 ,F y1 ,F x2 ,F y2 ,...,F xM ,F yM ] T is the traction force vector acting in the rectangular area corresponding to each node, F xm and F ym are the traction components of the mth node along the X and Y directions respectively; G is the coefficient matrix: and E is the Young’s modulus of the elastic substrate to which the cells adhere; Considering the inverse problem characteristics of cell traction inversion, the fourth neural network (N4) uses the classic Tikhonov regularization method to process the coefficient matrix G and obtain the regularized cell traction field f: f=RU formula (11) The displacement vector U is given by equation (5), and R is the new coefficient matrix: R = (G T G + η) -1 G T Equation (12) Where η is the regularization coefficient; the displacement field U determined by the above formulas (5)-(12) is used as the training data of the fourth neural network (N4), and the regularized cell traction force field f is used as the corresponding training label; In the process of training the fourth neural network (N4), the root mean square error is specified as the loss of the fourth neural network (N4) training, and the network parameters are optimized by the small batch stochastic gradient descent method; when the network training converges, the values ​​of the network parameters are recorded for subsequent calculations.

10. The method for quantifying cell traction force with ultra-high temporal and spatial resolution according to claim 1, characterized in that: The integrated displacement field (U, V) is input into the fourth neural network (N4) for prediction to obtain the distribution of the traction force field of the cell to be tested, including: The integrated displacement field (U, V) is input into the fourth neural network (N4) for prediction, and after being mapped by the fourth neural network (N4), the distribution of the traction force field of the cell to be tested is obtained and output.

11. An ultra-high spatiotemporal resolution cell traction force quantification device, using the ultra-high spatiotemporal resolution cell traction force quantification method according to any one of claims 1 to 10, characterized in that: include: A neural network construction and training module, used to respectively construct and train a first neural network (N1), a second neural network (N2), a third neural network (N3) and a fourth neural network (N4); a displacement recording module, for predicting the displacement of each sampling point using the trained first neural network (N1), the second neural network (N2) and the third neural network (N3) respectively in the fluorescence image of the fluorescent marker before digestion of the tested cells and the fluorescence image of the fluorescent marker after digestion of the tested cells, and integrating the displacement results predicted by the first neural network (N1), the second neural network (N2) and the third neural network (N3), and recording the current displacement field (U, V) composed of the displacements of all sampling points; and A cell traction force field calculation module is used to input the integrated displacement field (U, V) into the fourth neural network (N4) for prediction to obtain the distribution of the traction force field of the cell to be measured; The fluorescence images of the fluorescent markers of the cells to be tested before digestion and the fluorescence images of the fluorescent markers of the cells to be tested after digestion are respectively predicted by the trained first neural network (N1), the second neural network (N2) and the third neural network (N3) for displacement of each sampling point, including: The fluorescence images of the fluorescent markers of the cells to be tested before digestion and the fluorescence images of the fluorescent markers of the cells to be tested after digestion are input into the trained first neural network (N1) for prediction, and a displacement field (U1, V1) composed of the displacement values ​​of all sampling points is obtained through the mapping of the first neural network (N1), wherein U1 is the displacement of the displacement field along the X direction, and V1 is the displacement of the displacement field along the Y direction; The fluorescence images of the fluorescent markers of the cells to be tested before digestion and the fluorescence images of the fluorescent markers of the cells to be tested after digestion are input into the trained second neural network (N2) for prediction. Through the mapping of the second neural network (N2), a displacement field (U2, V2) composed of the displacement values ​​of all sampling points is obtained, wherein U2 is the displacement of the displacement field along the X direction, and V2 is the displacement of the displacement field along the Y direction; The fluorescence images of the fluorescent markers before and after digestion of the cells to be tested are input into the trained third neural network (N3) for prediction. Through the mapping of the third neural network (N3), a displacement field (U3, V3) composed of the displacement values ​​of all sampling points is obtained, wherein U3 is the displacement of the displacement field along the X direction, and V3 is the displacement of the displacement field along the Y direction; The integration of the displacement results predicted by the first neural network (N1), the second neural network (N2) and the third neural network (N3) includes: For the jth sampling point, if Pixel or V1 (j) >1 pixel, then take V (j) =V1 (j) ,otherwise if Pixel or Pixels, then take otherwise Pick in is the prediction result of the first neural network (N1) for the displacement of the jth sampling point along the X direction, V1 (j) is the prediction result of the first neural network (N1) for the displacement of the jth sampling point along the Y direction, is the prediction result of the second neural network (N2) for the displacement of the jth sampling point along the X direction, is the prediction result of the second neural network (N2) for the displacement of the jth sampling point along the Y direction, is the prediction result of the third neural network (N3) for the displacement of the jth sampling point along the X direction, is the prediction result of the third neural network (N3) for the displacement of the jth sampling point along the Y direction, U (j) is the displacement of the jth sampling point recorded along the X direction after integration, V (j) is the displacement of the jth sampling point recorded along the Y direction after integration; The above judgment is repeated for all J sampling points to obtain the integrated full-field displacement (U, V).

12. An electronic device comprising: processor; A memory storing a computer executable program, which, when executed by the processor, enables the processor to perform the ultra-high spatiotemporal resolution cell traction force quantification method according to any one of claims 1 to 10.

13. A storage medium containing computer executable instructions, characterized in that: When the computer executable instructions are executed by a processor, the ultra-high spatiotemporal resolution cell traction force quantification method according to any one of claims 1 to 10 is implemented.

14. A computer program product, characterized in that include: Computer executable instructions, when executed by a processor, are used to implement the ultra-high spatiotemporal resolution cell traction force quantification method according to any one of claims 1 to 10.

Images