Liposome attribute evaluation method and device, electronic equipment and storage medium

Abstract

The invention relates to a lipidosome attribute evaluation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a lipidosome image comprising at least one lipidosome, carrying out the encoding of the lipidosome image through an image encoder and a position encoder, and obtaining a corresponding image code and a corresponding position code; and decoding based on the image code and the position code through a decoder to obtain a mask image for marking the position of the liposome in the liposome image. And calculating the appearance attribute of the lipidosome in the lipidosome image according to the position of the lipidosome in the mask image and the pixel size representing the physical size represented by each pixel in the lipidosome image. According to the method, the position of the liposome in the image is automatically identified through the encoder and the decoder, the appearance attribute of the liposome in the image is further calculated, appearance attribute evaluation can be rapidly and accurately carried out on a large number of liposomes, the evaluation method is realized based on the image pixel position, and the finally obtained appearance attribute is high in precision.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a liposome property evaluation method, device, electronic device and storage medium. Background Art

[0002] Liposomes are currently the most widely studied nano drug carriers in drug delivery systems. They are closed vesicles composed of a lipid bilayer composed of phospholipids and cholesterol. Liposomes have a biological membrane-like structure and can encapsulate water-soluble and fat-soluble drugs. They are highly selective, non-toxic, non-immunogenic, and suitable for degradation in vivo. As drug carriers, liposomes have a high degree of targeting, can effectively protect the encapsulated drugs, can control and release drugs, significantly improve the therapeutic index of drugs, and reduce the adverse reactions of drugs. Therefore, the value of liposomes in drug development systems is very important. However, the different particle sizes and shapes of liposomes will affect the physical stability, biodistribution, drug release and targeting of liposomes. Therefore, accurate and rapid measurement of the size and shape of liposomes is an indispensable quality evaluation method. Summary of the invention

[0003] In view of this, the present disclosure proposes a liposome property evaluation method, device, electronic device and storage medium, aiming to accurately and quickly evaluate the size and shape of liposomes.

[0004] According to a first aspect of the present disclosure, a method for evaluating liposome properties is provided, the method comprising:

[0005] acquiring a liposome image including at least one liposome;

[0006] Encoding the liposome image by an image encoder to obtain a corresponding image code;

[0007] Encoding the liposome image by a position encoder to obtain a corresponding position code;

[0008] Decoding by a decoder based on the image code and the position code to obtain a mask image of the liposome image, wherein the mask image is used to mark the position of the liposome in the liposome image;

[0009] The appearance attributes of the liposomes in the liposome image are calculated according to the liposome positions in the mask image and the pixel size, and the pixel size is used to characterize the physical size represented by each pixel in the liposome image.

[0010] In a possible implementation, acquiring a liposome image including at least one liposome includes:

[0011] The corresponding liposome image is obtained by capturing an image of at least one liposome through a cryo-electron microscope.

[0012] In a possible implementation, encoding the liposome image by an image encoder to obtain a corresponding image code includes:

[0013] Downsampling the liposome image through a coding layer in the image encoder to obtain an image matrix;

[0014] The image matrix is ​​convolved through an embedding layer in the image encoder to obtain an image code in vector form.

[0015] In a possible implementation, encoding the liposome image by a position encoder to obtain a corresponding position code includes:

[0016] Determine the position coordinates corresponding to each pixel in the liposome image;

[0017] The position coordinates of each pixel are encoded by the position encoder to obtain a vector with the same dimension as the image encoding as the position code.

[0018] In a possible implementation, the decoder includes an attention mechanism and a transformer model, and the decoder performs decoding based on the image code and the position code to obtain a mask image of the liposome image, including:

[0019] Perform cross calculation on the image code and the position code through the attention mechanism to obtain a cross matrix;

[0020] The cross matrix is ​​upsampled by the transformer model to obtain a mask image with the same size as the liposome image.

[0021] In a possible implementation, calculating the appearance attributes of the liposomes in the liposome image according to the liposome positions and pixel sizes in the mask image includes:

[0022] Calculating the particle size of the liposome according to the liposome position in the mask image and the pixel size of the liposome image;

[0023] Calculating the circularity of the liposome according to the position of the liposome in the mask image, wherein the circularity represents the proportion of circular liposomes included in the liposome image;

[0024] The appearance attributes corresponding to the liposome image are determined according to the particle size and circularity of the liposome.

[0025] In a possible implementation, the calculating the particle size of the liposome according to the liposome position in the mask image and the pixel size of the liposome image includes:

[0026] Determining the long side size of each liposome according to the position of the liposome in the mask image;

[0027] Determine the particle size of the liposome according to the product of the long side dimension of each liposome and the pixel size;

[0028] The mean value of each liposome particle size is calculated to obtain the liposome particle size corresponding to the liposome image.

[0029] In a possible implementation, the calculating the circularity of the liposome according to the position of the liposome in the mask image includes:

[0030] Determining the long side size and the short side size of each liposome according to the liposome position in the mask image;

[0031] Determining whether each of the liposomes is circular according to the long side size and the short side size of each of the liposomes;

[0032] The proportion of circular liposomes included in the liposome image is calculated to obtain the corresponding circularity ratio.

[0033] In a possible implementation, determining whether each of the liposomes is circular according to the long side size and the short side size of each of the liposomes comprises:

[0034] For each of the liposomes, calculating the ratio of the difference between the long side dimension and the short side dimension to the long side dimension;

[0035] In response to the ratio being less than a preset threshold, determining that the liposome is round;

[0036] In response to the ratio being greater than or equal to the threshold, it is determined that the liposome is not round.

[0037] According to a second aspect of the present disclosure, a liposome property evaluation device is provided, the device comprising:

[0038] An image acquisition module, used for acquiring a liposome image including at least one liposome;

[0039] A first encoding module, used for encoding the liposome image through an image encoder to obtain a corresponding image code;

[0040] A second encoding module is used to encode the liposome image through a position encoder to obtain a corresponding position code;

[0041] A decoding module, used for decoding by a decoder based on the image code and the position code to obtain a mask image of the liposome image, wherein the mask image is used for marking the position of the liposome in the liposome image;

[0042] The attribute evaluation module is used to calculate the appearance attributes of the liposomes in the liposome image according to the liposome positions in the mask image and the pixel size, wherein the pixel size is used to characterize the physical size represented by each pixel in the liposome image.

[0043] In a possible implementation, the image acquisition module is further used to:

[0044] The corresponding liposome image is obtained by capturing an image of at least one liposome through a cryo-electron microscope.

[0045] In a possible implementation, the first encoding module is further configured to:

[0046] Downsampling the liposome image through a coding layer in the image encoder to obtain an image matrix;

[0047] The image matrix is ​​convolved through an embedding layer in the image encoder to obtain an image code in vector form.

[0048] In a possible implementation, the second encoding module is further configured to:

[0049] Determine the position coordinates corresponding to each pixel in the liposome image;

[0050] The position coordinates of each pixel are encoded by the position encoder to obtain a vector with the same dimension as the image encoding as the position code.

[0051] In a possible implementation, the decoder includes an attention mechanism and a transformer model, and the decoding module is further used to:

[0052] Perform cross calculation on the image code and the position code through the attention mechanism to obtain a cross matrix;

[0053] The cross matrix is ​​upsampled by the transformer model to obtain a mask image with the same size as the liposome image.

[0054] In a possible implementation, the attribute evaluation module is further used to:

[0055] Calculating the particle size of the liposome according to the liposome position in the mask image and the pixel size of the liposome image;

[0056] Calculating the circularity of the liposome according to the position of the liposome in the mask image, wherein the circularity represents the proportion of circular liposomes included in the liposome image;

[0057] The appearance attributes corresponding to the liposome image are determined according to the particle size and circularity of the liposome.

[0058] In a possible implementation, the attribute evaluation module is further used to:

[0059] Determining the long side size of each liposome according to the position of the liposome in the mask image;

[0060] Determine the particle size of the liposome according to the product of the long side dimension of each liposome and the pixel size;

[0061] The mean value of each liposome particle size is calculated to obtain the liposome particle size corresponding to the liposome image.

[0062] In a possible implementation, the attribute evaluation module is further used to:

[0063] Determining the long side size and the short side size of each liposome according to the liposome position in the mask image;

[0064] Determining whether each of the liposomes is circular according to the long side size and the short side size of each of the liposomes;

[0065] The proportion of circular liposomes included in the liposome image is calculated to obtain the corresponding circularity ratio.

[0066] In a possible implementation, the attribute evaluation module is further used to:

[0067] For each of the liposomes, calculating the ratio of the difference between the long side dimension and the short side dimension to the long side dimension;

[0068] In response to the ratio being less than a preset threshold, determining that the liposome is round;

[0069] In response to the ratio being greater than or equal to the threshold, it is determined that the liposome is not round.

[0070] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0071] According to a fourth aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0072] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0073] In an embodiment of the present disclosure, a liposome image including at least one liposome is obtained and the liposome image is encoded by an image encoder and a position encoder respectively to obtain a corresponding image code and a corresponding position code. A decoder is used to decode based on the image code and the position code to obtain a mask image for marking the position of the liposome in the liposome image. The appearance attributes of the liposome in the liposome image are calculated according to the position of the liposome in the mask image and the pixel size representing the physical size represented by each pixel in the liposome image. The present disclosure automatically identifies the position of the liposome in the image through an encoder and a decoder, and then calculates the appearance attributes of the liposome in the image, which can quickly and accurately evaluate the appearance attributes of a large number of liposomes, and the evaluation method is implemented based on the image pixel position, and the appearance attributes finally obtained are highly accurate.

[0074] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0076] Figure 1 A flow chart showing a method for evaluating liposome properties according to an embodiment of the present disclosure;

[0077] Figure 2 A schematic diagram showing a decoder structure according to an embodiment of the present disclosure;

[0078] Figure 3 A schematic diagram showing the long side and short side of a liposome according to an embodiment of the present disclosure;

[0079] Figure 4 A schematic diagram showing a liposome property evaluation process according to an embodiment of the present disclosure;

[0080] Figure 5 A schematic diagram showing a liposome property evaluation device according to an embodiment of the present disclosure;

[0081] Figure 6 A schematic diagram showing an electronic device according to an embodiment of the present disclosure;

[0082] Figure 7 A schematic diagram showing another electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0083] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0084] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0085] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0086] The liposome property evaluation method of the embodiment of the present disclosure can be performed by an electronic device such as a terminal device or a server. Among them, the terminal device can be any fixed or mobile terminal such as a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The server can be a single server or a server cluster composed of multiple servers. Any electronic device can implement the liposome property evaluation method of the embodiment of the present disclosure by calling a computer-readable instruction stored in a memory through a processor.

[0087] Figure 1 A flow chart of a liposome property evaluation method according to an embodiment of the present disclosure is shown. Figure 1 As shown, a liposome property evaluation method on the first user side of the embodiment of the present disclosure may include the following steps S10-S50.

[0088] Step S10: Acquire a liposome image including at least one liposome.

[0089] In a possible implementation, the electronic device may first acquire a liposome image, which includes at least one liposome, so as to evaluate the properties of the liposome included therein based on the liposome image. Optionally, the electronic device may acquire the liposome image by receiving an image acquired by an image acquisition device of another device, or may control a connected image acquisition device to directly acquire an image of at least one liposome to obtain a liposome image. Since liposomes are nanoscale micro drug carriers, image acquisition may be performed by an image acquisition device such as a cryo-electron microscope that can acquire the structure of biological macromolecules. That is, the electronic device may acquire the liposome image by acquiring an image of at least one liposome through a cryo-electron microscope to obtain a corresponding liposome image.

[0090] Step S20: Encode the liposome image by an image encoder to obtain a corresponding image code.

[0091] In a possible implementation, after acquiring the liposome image, the electronic device first encodes the content dimension of the liposome image through an image encoder to obtain the image code corresponding to the liposome image, and the image code can be used to characterize the image content information in the liposome image. Optionally, the image encoder of the embodiment of the present disclosure may include a coding layer and an embedding layer, and the liposome image can be input into the coding layer and the embedding layer in sequence, so as to realize the coding of the liposome image through the coding layer and the embedding layer together to obtain the corresponding image code. Among them, the coding layer is used to downsample the image, and the embedding layer is used to reduce the number of channels of the image. That is, the electronic device can downsample the liposome image through the coding layer in the image encoder to obtain an image matrix. Then, the image matrix is ​​convolved through the embedding layer in the image encoder to obtain the image code in vector form.

[0092] For example, when the size of the liposome image is 1024×1024, the encoding layer in the encoder can perform 16x downsampling on the input image to obtain an image matrix of size 64×64. Further, the 64×64 image matrix output by the encoding layer is convolved using a 1×1 convolution kernel through the embedding layer to obtain a vector with a reduced number of channels as the image code, and the vector includes 256 dimensions.

[0093] Step S30: Encode the liposome image by using a position encoder to obtain a corresponding position code.

[0094] In a possible implementation, after acquiring the liposome image, the electronic device also encodes the pixel position dimension of the liposome image through a position encoder to obtain a position code corresponding to the liposome image, and the position code can be the same as the position characterizing each pixel in the liposome image. Optionally, the process of position coding and the process of image coding in the embodiment of the present disclosure do not have a fixed order and can be performed simultaneously or successively.

[0095] Optionally, when encoding the pixel position dimension, the electronic device may first determine the position coordinates corresponding to each pixel in the liposome image. Then, the position coordinates of each pixel are encoded by the position encoder to obtain a vector with the same dimension as the image encoding as the position encoding. Among them, the position coordinates of each pixel can be represented by (x, y), and the position encoding process can input the position coordinates of each pixel into the formula:

[0096]

[0097] Among them, v represents the (x, y) coordinates, ai can be a value of 0-1, bi is a vector randomly sampled according to Gaussian distribution, T represents the transpose of the bi vector, and m represents the dimension of the vector. The position coordinates of each pixel can be transformed using the vector obtained by random sampling, and finally a vector with the same dimension as the image encoding is obtained as the position encoding.

[0098] Step S40: decoding by a decoder based on the image code and the position code to obtain a mask image of the liposome image.

[0099] In a possible implementation, the electronic device encodes the liposome image at the image level and the position level respectively to obtain the corresponding image code and position code, and then decodes the image code and the position code through a decoder to obtain a mask image corresponding to the liposome image. The mask image corresponding to the liposome image is used to mark the position of the liposome in the liposome image. Exemplarily, the size of the mask image is the same as the size of the liposome image, and the pixel value of the mask image is 0 or 1, the pixel value 0 indicates that the corresponding position in the liposome image is not the position where the liposome is located, and the pixel value 1 indicates that the corresponding position in the liposome image is the position where the liposome is located.

[0100] Optionally, the decoder of the embodiment of the present disclosure may include an attention mechanism and a transformer model. The method for determining the mask image corresponding to the liposome image through the decoder can be to first cross-calculate the image encoding and the position encoding through the attention mechanism to obtain a cross matrix, and then up-sample the cross matrix through the transformer model to obtain a mask image with the same size as the liposome image.

[0101] Figure 2 A schematic diagram of a decoder structure according to an embodiment of the present disclosure is shown. Figure 2 As shown in the figure, the attention mechanism in the decoder also includes an image attention layer (Image to point attn), a position attention layer (point toimage attn) and a multi-layer perceptron module (MLP). The attention mechanism of the decoder can perform cross calculations on the image encoding (image embedding) and the position encoding (point embedding) based on the above cross-set multi-layer structure to obtain a cross matrix, and then upsample the cross matrix through the transformer model to obtain a mask image with the same size as the liposome image. The cross calculation process can be implemented by the following formula:

[0102]

[0103] Among them, when calculating through the image attention layer and the position attention layer, the image encoding is Q, the position encoding is K, V, and d is the encoding dimension. The multi-layer perceptron can be a multi-layer fully connected network model with pre-set input layer dimensions, hidden layer dimensions, number of layers, and output layer dimensions. The transformer model (2X Conv Trans) parameters can also be pre-set, for example, the transposed convolution used to enlarge the output image embedding is set to 2×2, the stride is 2, the output channel size is 64 and 32, and the GELU activation function is used.

[0104] Furthermore, the disclosed embodiment can also mark the area where the liposomes are located in the liposome image according to the obtained mask image, that is, the liposomes are framed by a square image frame. Optionally, the liposome image with the liposome area framed can be displayed by a display device, and when an error occurs in marking the liposome area, the liposome image can be manually edited. For example, if a certain liposome is not recognized, the position of the area can be determined, and the area can be intercepted to identify the liposome position again.

[0105] Step S50: Calculate the appearance attributes of the liposomes in the liposome image according to the positions and pixel sizes of the liposomes in the mask image.

[0106] In a possible implementation, after determining the mask image, the electronic device calculates the appearance attributes of the liposomes in the liposome image according to the liposome positions in the mask image and the pixel size. The pixel size is used to characterize the physical size represented by each pixel in the liposome image. The appearance attributes may include the size of the liposomes in the liposome image and whether the liposomes are round. Optionally, when a plurality of liposomes are included in the liposome image, the average size corresponding to the plurality of liposomes and the probability of a round liposome among the plurality of liposomes may be determined as the appearance attributes corresponding to all the liposomes in the liposome image.

[0107] Optionally, when the electronic device determines the appearance attributes corresponding to all liposomes in the liposome image, the particle size of the liposome can be calculated according to the liposome position in the mask image and the pixel size of the liposome image. And the return to roundness of the liposome is calculated according to the liposome position in the mask image, and the return to roundness represents the proportion of circular liposomes included in the liposome image. Then the appearance attributes corresponding to the liposome image are determined according to the particle size and return to roundness of the liposome. Among them, the particle size can be determined according to the average value calculated according to the particle size of each liposome. That is, when the liposome is marked by a square image frame, the long side size of the square image frame of each liposome can be determined according to the liposome position in the mask image. The particle size of the liposome is determined according to the product of the long side size of each liposome and the pixel size. Finally, the mean value of the particle size of each liposome is calculated to obtain the liposome particle size corresponding to the liposome image. At the same time, the long side size and short side size of each liposome can also be determined according to the liposome position in the mask image. According to the long side size and short side size of each liposome, it is determined whether each liposome is circular. The proportion of circular liposomes included in the liposome image is calculated to obtain the corresponding return to roundness.

[0108] Figure 3 Schematic diagram showing the long side and short side of a liposome according to an embodiment of the present disclosure. Figure 3 As shown in the figure, the square image frame for selecting the position of the liposome may include a long side and a short side, and the difference between the long side and the short side may be used to determine whether each liposome is circular. That is, for each liposome, the difference between the long side and the short side and the ratio of the long side may be calculated. If the ratio is less than a preset threshold, the liposome is determined to be circular, and if the ratio is greater than or equal to the threshold, the liposome is determined to be not circular.

[0109] After obtaining the particle size and shape of each liposome, the appearance attributes corresponding to each liposome can be directly determined. At the same time, the mean and circularity can be calculated based on the particle size and shape of all liposomes in the liposome image to obtain the overall appearance attributes of the liposome image.

[0110] Figure 4A schematic diagram of a liposome property evaluation process according to an embodiment of the present disclosure is shown. Figure 4 As shown, after determining a liposome image including at least one liposome, the electronic device of the disclosed embodiment simultaneously encodes the liposome image through an image encoder and a position encoder to obtain corresponding image codes and position codes. Then, both the image code and the position code are input into a decoder, which decodes the image code and the position code to obtain a mask image that marks the position of the liposome in the liposome image.

[0111] Based on the above technical features, the disclosed embodiment can automatically identify the position of the liposome in the image through the encoder and the decoder, and then calculate the appearance attributes of the liposome in the image according to the position of the liposome and the preset pixel size, so as to evaluate the quality of the liposome through the appearance attributes. This method is simple and efficient, and can quickly and accurately evaluate the appearance attributes of a large number of liposomes. At the same time, the liposome appearance attribute evaluation method performs recognition and calculation based on the image pixel position to perform pixel-level recognition of the size and shape of the liposome, and the appearance attributes obtained are highly accurate.

[0112] Figure 5 A schematic diagram of a liposome property evaluation device according to an embodiment of the present disclosure is shown. Figure 5 As shown, the liposome property evaluation device of the embodiment of the present disclosure may include:

[0113] An image acquisition module 50, used to acquire a liposome image including at least one liposome;

[0114] A first encoding module 51 is used to encode the liposome image through an image encoder to obtain a corresponding image code;

[0115] A second encoding module 52 is used to encode the liposome image through a position encoder to obtain a corresponding position code;

[0116] A decoding module 53, configured to decode the image code and the position code through a decoder to obtain a mask image of the liposome image, wherein the mask image is used to mark the position of the liposome in the liposome image;

[0117] The attribute evaluation module 54 is used to calculate the appearance attributes of the liposomes in the liposome image according to the liposome positions in the mask image and the pixel size, wherein the pixel size is used to characterize the physical size represented by each pixel in the liposome image.

[0118] In a possible implementation, the image acquisition module 50 is further used to:

[0119] The corresponding liposome image is obtained by capturing an image of at least one liposome through a cryo-electron microscope.

[0120] In a possible implementation, the first encoding module 51 is further configured to:

[0121] Downsampling the liposome image through a coding layer in the image encoder to obtain an image matrix;

[0122] The image matrix is ​​convolved through an embedding layer in the image encoder to obtain an image code in vector form.

[0123] In a possible implementation, the second encoding module 52 is further configured to:

[0124] Determine the position coordinates corresponding to each pixel in the liposome image;

[0125] The position coordinates of each pixel are encoded by the position encoder to obtain a vector with the same dimension as the image encoding as the position code.

[0126] In a possible implementation, the decoder includes an attention mechanism and a transformer model, and the decoding module 53 is further used to:

[0127] Perform cross calculation on the image code and the position code through the attention mechanism to obtain a cross matrix;

[0128] The cross matrix is ​​upsampled by the transformer model to obtain a mask image with the same size as the liposome image.

[0129] In a possible implementation, the attribute evaluation module 54 is further configured to:

[0130] Calculating the particle size of the liposome according to the liposome position in the mask image and the pixel size of the liposome image;

[0131] Calculating the circularity of the liposome according to the position of the liposome in the mask image, wherein the circularity represents the proportion of circular liposomes included in the liposome image;

[0132] The appearance attributes corresponding to the liposome image are determined according to the particle size and circularity of the liposome.

[0133] In a possible implementation, the attribute evaluation module 54 is further configured to:

[0134] Determining the long side size of each liposome according to the position of the liposome in the mask image;

[0135] Determine the particle size of the liposome according to the product of the long side dimension of each liposome and the pixel size;

[0136] The mean value of each liposome particle size is calculated to obtain the liposome particle size corresponding to the liposome image.

[0137] In a possible implementation, the attribute evaluation module 54 is further configured to:

[0138] Determining the long side size and the short side size of each liposome according to the liposome position in the mask image;

[0139] Determining whether each of the liposomes is circular according to the long side size and the short side size of each of the liposomes;

[0140] The proportion of circular liposomes included in the liposome image is calculated to obtain the corresponding circularity ratio.

[0141] In a possible implementation, the attribute evaluation module 54 is further configured to:

[0142] For each of the liposomes, calculating the ratio of the difference between the long side dimension and the short side dimension to the long side dimension;

[0143] In response to the ratio being less than a preset threshold, determining that the liposome is round;

[0144] In response to the ratio being greater than or equal to the threshold, it is determined that the liposome is not round.

[0145] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0146] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0147] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0148] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0149] Figure 6 A schematic diagram of an electronic device 800 according to an embodiment of the present disclosure is shown. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0150] Reference Figure 6 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0151] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0152] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0153] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0154] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0155] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0156] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0157] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0158] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0159] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0160] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above method.

[0161] Figure 7 1 is a schematic diagram showing another electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or a terminal device. Figure 7 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0162] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

[0163] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0164] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0165] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0166] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0167] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0168] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0169] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0170] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0171] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0172] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for evaluating liposome properties, characterized in that: The method comprises: acquiring a liposome image including at least one liposome; Encoding the liposome image by an image encoder to obtain a corresponding image code; Encoding the liposome image by a position encoder to obtain a corresponding position code; Decoding by a decoder based on the image code and the position code to obtain a mask image of the liposome image, wherein the mask image is used to mark the position of the liposome in the liposome image; The appearance attributes of the liposomes in the liposome image are calculated according to the liposome positions in the mask image and the pixel size, and the pixel size is used to characterize the physical size represented by each pixel in the liposome image.

2. The method according to claim 1, characterized in that The step of acquiring a liposome image including at least one liposome comprises: The corresponding liposome image is obtained by capturing an image of at least one liposome through a cryo-electron microscope.

3. The method according to claim 1 or 2, characterized in that: The step of encoding the liposome image by an image encoder to obtain a corresponding image code comprises: Downsampling the liposome image through a coding layer in the image encoder to obtain an image matrix; The image matrix is ​​convolved through an embedding layer in the image encoder to obtain an image code in vector form.

4. The method according to any one of claims 1 to 3, characterized in that: The step of encoding the liposome image by a position encoder to obtain a corresponding position code comprises: Determine the position coordinates corresponding to each pixel in the liposome image; The position coordinates of each pixel are encoded by the position encoder to obtain a vector with the same dimension as the image encoding as the position code.

5. The method according to any one of claims 1 to 4, characterized in that: The decoder includes an attention mechanism and a transformer model, and the decoder performs decoding based on the image code and the position code to obtain a mask image of the liposome image, including: Perform cross calculation on the image code and the position code through the attention mechanism to obtain a cross matrix; The cross matrix is ​​upsampled by the transformer model to obtain a mask image with the same size as the liposome image.

6. The method according to any one of claims 1 to 5, characterized in that: The step of calculating the appearance attributes of the liposomes in the liposome image according to the liposome positions and pixel sizes in the mask image includes: Calculating the particle size of the liposome according to the liposome position in the mask image and the pixel size of the liposome image; Calculating the circularity of the liposome according to the position of the liposome in the mask image, wherein the circularity represents the proportion of circular liposomes included in the liposome image; The appearance attributes corresponding to the liposome image are determined according to the particle size and circularity of the liposome.

7. The method according to claim 6, characterized in that The step of calculating the particle size of the liposome according to the liposome position in the mask image and the pixel size of the liposome image comprises: Determining the long side size of each liposome according to the position of the liposome in the mask image; Determine the particle size of the liposome according to the product of the long side dimension of each liposome and the pixel size; The mean value of each liposome particle size is calculated to obtain the liposome particle size corresponding to the liposome image.

8. The method according to claim 6 or 7, characterized in that: The step of calculating the circularity of the liposome according to the position of the liposome in the mask image comprises: Determining the long side size and the short side size of each liposome according to the liposome position in the mask image; Determining whether each of the liposomes is circular according to the long side size and the short side size of each of the liposomes; The proportion of circular liposomes included in the liposome image is calculated to obtain the corresponding circularity ratio.

9. The method according to claim 8, characterized in that The step of determining whether each of the liposomes is circular according to the long side size and the short side size of each of the liposomes comprises: For each of the liposomes, calculating the ratio of the difference between the long side dimension and the short side dimension to the long side dimension; In response to the ratio being less than a preset threshold, determining that the liposome is round; In response to the ratio being greater than or equal to the threshold, it is determined that the liposome is not round.

10. A liposome property evaluation device, characterized in that: The device comprises: An image acquisition module, used for acquiring a liposome image including at least one liposome; A first encoding module, used for encoding the liposome image through an image encoder to obtain a corresponding image code; A second encoding module is used to encode the liposome image through a position encoder to obtain a corresponding position code; A decoding module, used for decoding by a decoder based on the image code and the position code to obtain a mask image of the liposome image, wherein the mask image is used for marking the position of the liposome in the liposome image; The attribute evaluation module is used to calculate the appearance attributes of the liposomes in the liposome image according to the liposome positions in the mask image and the pixel size, wherein the pixel size is used to characterize the physical size represented by each pixel in the liposome image.

11. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the method described in any one of claims 1 to 9 when executing the instructions stored in the memory.

12. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images