Remote real-time analysis and monitoring system for cell growth

Abstract

The invention provides a cell growth remote real-time analysis and monitoring method, which comprises the following steps: acquiring at least two cell images of a to-be-monitored area acquired at a preset acquisition interval, and inputting the cell images into an instance segmentation module obtained through training to obtain a first cell feature, fusing the first cell feature and the second cell feature to obtain a cell image feature of the current cell image; and arranging the cell image features of the plurality of cell images according to an acquisition time sequence to form a cell feature sequence, and inputting the cell feature sequence into a prediction module obtained through training to obtain a predicted cell number and a predicted fusion degree of the to-be-monitored cells. According to the scheme, by constructing the instance segmentation module and the prediction module, the cell number and the fusion degree are predicted by performing instance segmentation on the cell image and combining the environmental information in the cell growth process.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a remote real-time analysis and monitoring system for cell growth. Background Art

[0002] With the development of modern science, in vitro cell culture technology has become an indispensable research tool in many fields, such as genetics, immunology, oncology, virology, molecular biology and other fields. In vitro cell culture refers to placing cells collected from in vivo tissues under sterile, suitable temperature, humidity and rich nutrition conditions to enable them to grow, reproduce and maintain their structure and function. The most critical thing about in vitro cell culture is the maintenance of cell viability, so experimenters need to observe and detect the morphology and growth status of cells in a long-term dynamic manner.

[0003] When observing cells, experimenters usually take the cells out of the incubator and place them under an inverted microscope for observation. However, the direct observation method exposes the cell culture dish to the air, increasing the risk of cell contamination. In addition, the cells may also be stressed by changes in ambient temperature, thereby affecting the cell growth state and reducing cell activity. In addition, cell number and cell fusion are also important indicators for evaluating cell viability and different experimental conditions. They can not only help experimenters deeply understand the molecular mechanisms at the cellular level, but also provide important guidance for the prediction of disease progression and the customization of treatment plans. The main methods of cell counting currently include manual counting, cell counter counting and flow cytometry, but all three methods require the use of trypsin or cell scraping to peel the cells off the bottom of the container, make a cell suspension and dilute it to an appropriate multiple for observation. In the manual counting method, the cell suspension needs to be stained with trypan blue, and the cells stained blue cannot be counted during the counting. After the counting is completed, the cell counting plate needs to be cleaned. The operation is cumbersome and the human interpretation error is large. In the cell counter counting method, it is impossible to distinguish between dead cells and living cells, and the counting accuracy is poor. When using flow cytometry for cell counting, it is efficient and fast, and the results are accurate, but the instrument is expensive, the maintenance cost is high, and professional operators are required. In addition, the analysis of cell fusion is usually manually interpreted based on the experience of the experimenter, which has large errors and low accuracy.

[0004] With the development of machine vision technology, although the existing technology can calculate cell counting and cell fusion through neural networks, it is only effective in detecting scattered cells, but poor in detecting connected cells. In addition, the existing neural network only considers the current cell status during detection, and lacks analysis and early warning of the future development status of the cells.

[0005] In summary, there is an urgent need for a method that can monitor the cell status in real time without taking the cells out of the incubator and can accurately analyze the future development status of the current cells. Summary of the invention

[0006] The embodiment of the present application provides a method and device for remote real-time analysis and monitoring of cell growth, which predicts the number and degree of cell fusion by constructing an instance segmentation module and a prediction module, performing instance segmentation on cell images and combining environmental information during cell growth.

[0007] In a first aspect, an embodiment of the present application provides a method for remote real-time analysis and monitoring of cell growth, the method comprising:

[0008] Acquire at least two cell images of the area to be monitored acquired at a preset acquisition interval;

[0009] Inputting the cell image into the trained instance segmentation module to obtain a first cell feature corresponding to each cell image, fusing the first cell feature with the second cell feature to obtain a cell image feature of the current cell image, wherein the first cell feature includes cell type, cell number and cell fusion degree, and the second cell feature is an environmental feature of the growth environment corresponding to the current cell image;

[0010] The cell image features of multiple cell images are arranged in the order of acquisition time to form a cell feature sequence, and the cell feature sequence is input into the trained prediction module to obtain the predicted cell number and predicted fusion degree of the cells to be monitored.

[0011] In a second aspect, the present application provides a remote real-time analysis and monitoring device for cell growth, including a platform end, a device end, and a backend database:

[0012] The platform end is used to send operation instructions to the device end;

[0013] The device end includes a data acquisition module, a trained instance segmentation module and a trained prediction module, wherein the data acquisition module is used to obtain at least two cell images of the area to be monitored acquired at a preset acquisition interval, input the cell images into the trained instance segmentation module to obtain a first cell feature corresponding to each cell image, fuse the first cell feature and the second cell feature to obtain a cell image feature of the current cell image, wherein the first cell feature includes a cell category, a cell number and a cell fusion degree, and the second cell feature is an environmental feature of a growth environment corresponding to the current cell image, arrange the cell image features of multiple cell images in order of acquisition time to form a cell feature sequence, and input the cell feature sequence into the trained prediction module to obtain a predicted cell number and a predicted fusion degree of the cells to be monitored;

[0014] The backend database is used to store all information generated by the platform side and the device side, and is used for calling by the platform side and the device side.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute a method for remote real-time analysis and monitoring of cell growth.

[0016] In a fourth aspect, an embodiment of the present application provides a readable storage medium, wherein the readable storage medium stores a computer program, wherein the computer program includes a program code for controlling a process to execute a process, wherein the process includes a method for remote real-time analysis and monitoring of cell growth.

[0017] The main contributions and innovations of the present invention are as follows:

[0018] The embodiment of the present application obtains the cell category and cell number in the cell image by constructing an instance segmentation module, and the instance segmentation module in this scheme sets a dynamic segmentation threshold for each detection frame based on the mask area of ​​the detection frame with the highest confidence, thereby effectively avoiding the mistaken identification of multiple closely adjacent cells as a single large cell, or the mistaken segmentation of a single cell into multiple small cells, thereby improving the accuracy of segmentation; the prediction module in this scheme will combine the environmental information of each cell image during growth to make a comprehensive prediction, and through the prediction of multiple cell images of the same monitored area, the prediction accuracy of the prediction module for the monitored area can be dynamically improved, and with the accumulation of experimental data, the prediction module can self-optimize and provide increasingly accurate growth predictions.

[0019] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0021] Figure 1 is a flow chart of a method for remote real-time analysis and monitoring of cell growth according to an embodiment of the present application;

[0022] Figure 2 is a schematic diagram of a device for remote real-time analysis and monitoring of cell growth according to an embodiment of the present application;

[0023] Figure 3is a schematic diagram of using an attention mechanism to perform feature fusion according to an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of a prediction module according to an embodiment of the present application;

[0025] Figure 5 is a schematic diagram of a training process of an instance segmentation module and a prediction module according to an embodiment of the present application;

[0026] Figure 6 is a schematic diagram of an application flow of an instance segmentation module and a prediction module according to an embodiment of the present application;

[0027] Figure 7 is a schematic diagram of a main control unit according to an embodiment of the present application;

[0028] Figure 8 This is a schematic diagram of data interaction between a backend database and a front end according to an embodiment of the present application;

[0029] Fig. 9 This is a schematic diagram of data interaction between a device and a backend database according to an embodiment of the present application;

[0030] Fig.10 is a schematic diagram of data interaction between a main control unit and a lower position according to an embodiment of the present application;

[0031] Fig.11 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0033] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0034] Embodiment 1

[0035] The present application embodiment provides a remote real-time analysis and monitoring method for cell growth, which constructs an instance segmentation module and a prediction module, performs instance segmentation on cell images and combines environmental information during cell growth to predict the number and degree of cell fusion. Specifically, reference Figure 1 , the method for remote real-time analysis and monitoring of cell growth comprises:

[0036] Acquire at least two cell images of the area to be monitored acquired at a preset acquisition interval;

[0037] Inputting the cell image into the trained instance segmentation module to obtain a first cell feature corresponding to each cell image, fusing the first cell feature with the second cell feature to obtain a cell image feature of the current cell image, wherein the first cell feature includes cell type, cell number and cell fusion degree, and the second cell feature is an environmental feature of the growth environment corresponding to the current cell image;

[0038] The cell image features of multiple cell images are arranged in the order of acquisition time to form a cell feature sequence, and the cell feature sequence is input into the trained prediction module to obtain the predicted cell number and predicted fusion degree of the cells to be monitored.

[0039] It should be noted that the method for remote real-time analysis and monitoring of cell growth in this scheme is implemented by a remote real-time analysis and monitoring device for cell growth. Figure 2 As shown, including:

[0040] Platform side, device side and backend database;

[0041] The platform end is used to send operation instructions to the device end;

[0042] The device end includes a data acquisition module, a trained instance segmentation module and a trained prediction module, wherein the data acquisition module is used to obtain at least two cell images of cells to be monitored acquired at a preset acquisition interval, input the cell images into the trained instance segmentation module to obtain a first cell feature corresponding to each cell image, fuse the first cell feature and the second cell feature to obtain a cell image feature of the current cell image, wherein the first cell feature includes a cell category, a cell number and a cell fusion degree, and the second cell feature is an environmental feature of a growth environment corresponding to the current cell image, arrange the cell image features of multiple cell images in order of acquisition time to form a cell feature sequence, and input the cell feature sequence into the trained prediction module to obtain a predicted cell number and a predicted fusion degree of the cells to be monitored;

[0043] The backend database is used to store all information generated by the platform side and the device side, and is used for calling by the platform side and the device side.

[0044] About data acquisition module:

[0045] This solution uses a data acquisition module to acquire at least two cell images of cells to be monitored acquired at a preset acquisition interval. Specifically, in the step of "acquiring at least two cell images of cells to be monitored acquired at a preset acquisition interval", the cell images are pre-processed by any one or more of image brightness processing, image offset correction processing, image compression processing, and image recognition processing.

[0046] Specifically, in some embodiments, image brightness processing is performed on each cell image to improve cell visibility and resolution. The image brightness processing method is to optimize the brightness of each cell image to improve the contrast and brightness of the cell image until the segmentation requirements of the instance segmentation module are met.

[0047] In some embodiments, image offset correction is performed on each cell image to ensure the accuracy and consistency of the image. The image offset correction is performed by correcting the orientation of each cell image to a uniform orientation to eliminate the offset caused by device movement or lens shaking during the image acquisition process.

[0048] In some embodiments, image compression processing is performed on each cell image to compress the volume while ensuring image quality, thereby reducing storage volume.

[0049] In some embodiments, image recognition processing is performed on each cell image to facilitate cell identification. The image recognition processing means is to convert the pixel value of each cell image into RGB value, and normalize the R, G, and B values ​​of each cell image. Specifically, the normalization can be completed by dividing the R value, G value, and B value in the cell image by 255.

[0050] The data acquisition module of the present solution is used to collect cell images. In some embodiments, the data acquisition module includes a camera controlled by the device end. After receiving the acquisition instruction from the platform end, the device end controls the camera to collect cell images of the cells to be monitored according to a preset acquisition interval. The cell images are subjected to any one or more of the pre-processing of image brightness processing, image offset correction processing, image compression processing and image recognition processing on the device end, and then transmitted to the background database for storage.

[0051] In some embodiments, the preset acquisition interval in the device acquisition module is manually set. The preset acquisition interval in this solution is 1 hour, that is, the device acquisition module will automatically acquire an image every 1 hour.

[0052] About the instance segmentation module:

[0053] The example segmentation module in this scheme includes a detection unit, a segmentation unit and a classification unit. The detection unit records the position of each cell in the cell image with a detection frame. The segmentation unit is used to determine whether each pixel point in the detection frame is a cell pixel so as to segment the cell contour. The classification unit is used to identify the category of each cell, wherein dead cells are treated as a separate cell category.

[0054] In some embodiments, the instance segmentation module can adopt any intelligent model that can perform instance segmentation. In this solution, Yolov9 is used as the instance segmentation module.

[0055] Furthermore, a detection threshold is set, and detection frames with confidence greater than the detection threshold are retained. Then an initial segmentation threshold is set, and each pixel in each detection frame is segmented using the initial segmentation threshold. Pixels greater than the initial segmentation threshold are regarded as cell pixels, and pixels less than or equal to the initial segmentation threshold are regarded as non-cell pixels. The mask area composed of all cell pixels in each detection frame is calculated. Finally, the segmentation threshold of each detection frame is dynamically adjusted based on the mask area of ​​the detection frame with the highest confidence to obtain a dynamic threshold corresponding to each detection frame. Each detection frame is segmented based on the corresponding dynamic threshold to obtain the cell contour.

[0056] Specifically, the formula for dynamically adjusting the segmentation threshold of each detection frame is expressed as follows:

[0057] Tseg=Si / (Si+Smax)

[0058] Among them, Tseg is the dynamic threshold, Si is the mask area of ​​the current detection box under the initial segmentation threshold, and Smax is the mask area of ​​the detection box with the highest confidence.

[0059] Specifically, by dynamically adjusting the segmentation thresholds of different detection frames, it is possible to effectively avoid misidentifying multiple closely adjacent cells as a single large cell, or missegmenting a single cell into multiple small cells, thereby improving the accuracy of segmentation.

[0060] Specifically, when determining the type of cells, this solution will treat the dead cells as a separate cell type, and can issue an alarm when the number of dead cells reaches a certain level.

[0061] Specifically, category labels are used to represent cell categories, ranging from 0 to C, where C is the number of categories, numbers 0 to C-1 correspond to different types of surviving cells, and C is the number of all dead cells.

[0062] In some specific embodiments, the number of cells and the degree of fusion in the cell image can be obtained by analyzing the cell segmentation results, so as to be used for prediction in a subsequent prediction module.

[0063] In some specific embodiments, the training method of the instance segmentation module is:

[0064] A plurality of cell images marked with cell categories and a mask image corresponding to each cell image are obtained as training samples, wherein the mask image records the contour information of each cell, and the instance segmentation framework constructed in advance is trained with the training samples to obtain a trained instance segmentation network.

[0065] During the training process, in order to ensure the generalization of the instance segmentation network, the training samples are augmented by scaling, rotating, and changing the contrast.

[0066] In this scheme, the attention mechanism is used to assign weights to cell categories, cell numbers, cell fusion degrees, and environmental features, and the weighted sum is used to obtain cell image features. The fusion process using the attention mechanism is as follows: Figure 3 shown.

[0067] Specifically, by using the attention mechanism to assign weights to multiple types of features and fuse them, not only can the computational complexity of the prediction module be reduced, but the most powerful features for prediction can also be highlighted to enhance the accuracy of the prediction results.

[0068] In this scheme, the growth environment characteristics of cells include temperature, humidity, carbon dioxide concentration, etc. This scheme combines these multimodal data to make more accurate growth predictions, thereby identifying the impact of environmental factors on cell growth, and can also optimize the cell growth environment based on the prediction results.

[0069] Furthermore, in order to ensure the accuracy of the prediction, the first cell feature and the second cell feature are normalized.

[0070] In some embodiments, the upper limit of the number of cell images is calculated based on the ratio of the cell area in the cell image obtained by the instance segmentation module to the entire cell image, or the upper limit of the number of each cell image is set directly by manual setting, and the number of cells in the first cell feature is divided by the upper limit of the cell number to complete normalization.

[0071] Exemplarily, this solution uses a manual setting method to set the upper limit of the number of each cell image, and the upper limit of the number of cells in this solution is set to 400.

[0072] In some embodiments, the second cell characteristic includes temperature and humidity, and the normalization is performed by dividing the temperature by 100 and the relative humidity by 100.

[0073] About the prediction module:

[0074] The schematic diagram of the prediction module in this scheme is as follows Figure 4 As shown, the structure of the prediction module is three fully connected layers in series. The prediction time span is set, and the cell feature sequence is combined with the prediction time span and input into the prediction module to obtain the predicted cell number and predicted fusion degree within the prediction time span.

[0075] For example, when the prediction time span is set to 5 hours, the prediction results of the prediction module are the predicted cell number and predicted fusion within 5 hours. That is to say, if the acquisition time set in this scheme is to collect a cell image every 1 hour, then the result obtained is 5 prediction results within 5 hours.

[0076] Furthermore, the longest prediction time is set, and the prediction time span is normalized by the longest prediction time and then input into the prediction module. The formula is as follows:

[0077] Forecast time span / maximum forecast time

[0078] Among them, the prediction time span is the future time minus the current time. That is to say, in this scheme, when the time span is greater than the longest prediction time, that is, time span / longest prediction time ≥ 1, it exceeds the prediction ability of the prediction module and cannot be predicted.

[0079] In this scheme, the predicted number of cells will reach an upper limit, which is an artificially set value. For example, if the upper limit of the number of cells set in this scheme is 400, the number of cells predicted by the prediction module will not be greater than 400.

[0080] In some specific embodiments, the training process diagram of the instance segmentation module and the prediction module is as follows: Figure 5 As shown, the application flow diagram of the instance segmentation module and the prediction module is as follows Figure 6 shown.

[0081] The predicted cell number and predicted fusion degree obtained by the prediction module can issue an early warning before an abnormal situation occurs, so as to adjust the experimental conditions in advance, such as the ratio of nutrient solution or fine-tuning of environmental parameters. This prediction model can significantly improve the success rate of the experiment and reduce the experimental cycle and cost.

[0082] In the remote real-time analysis and monitoring device for cell growth, the platform side is built through a Web application. Users can access the platform side interface through a browser to operate. The platform side has user verification and authority management functions to ensure system security. The platform side also provides a graphical interface to display real-time data, analysis results and operation buttons for easy user operation.

[0083] In some specific embodiments, the platform provides a user name and password input box, as well as a submit button, and provides a user authentication function to ensure that only authorized users can access the system. The user fills in the user name and password and sends it to the server through an https request. After receiving the request, the backend queries the corresponding user name and password in the database for matching verification. If the verification is successful, a login success message is returned, otherwise a failure message is returned.

[0084] Specifically, the operation instructions issued by the platform include monitoring points, shooting time intervals, etc. An editable table or form is provided on the platform, and users are allowed to enter or modify parameter values ​​in it. The back end provides an API interface for the front end to call. The API interface is used to obtain the current parameter value or update the parameter value.

[0085] In this solution, the device receives various operation instructions sent by the platform through the main control unit. The schematic diagram of the main control unit is as follows: Figure 7 As shown, the main control unit is loaded with an instance segmentation module and a prediction module, and is responsible for performing various preprocessing on the cell images collected by the image acquisition module. The device side drives the motor through the lower computer to control the camera to move to the designated location and collect data at a preset collection interval. At the same time, the lower computer also obtains environmental features through various sensors such as temperature and humidity and returns them to the main control unit for various calculations and predictions.

[0086] Specifically, after the platform sends the start monitoring command, the main control processing unit on the device side controls the motor module to move the camera to the specified position according to the platform's command and collects images. The main control processing unit receives the collected image data, applies image processing and analysis algorithms, optimizes the image quality, counts the number of cells and the degree of fusion, and generates quantitative data. For existing cell models in the system, the system uses image segmentation technology to extract the cell area in the image, and then automatically identifies and counts the number of cells and the degree of fusion.

[0087] In this solution, the predicted cell number and predicted fusion degree are called in the background database, and the predicted cell number and predicted fusion degree are generated into an image, a scatter plot, a curve graph or a time-lapse video and displayed on the platform.

[0088] Specifically, by generating images, scatter plots, curve graphs or time-lapse videos based on the predicted cell number and predicted fusion degree, customers can intuitively see the experimental results. In the field of scientific research, it provides visual data for researchers in the fields of cell biology, drug screening, disease mechanism research, etc., which helps promote the progress of related research; in the field of education, the system can be used as a teaching aid to help students understand the complex processes of cell biology more intuitively.

[0089] In some specific embodiments, the data interaction between the backend database and the front end is as follows: Figure 8 As shown, the steps include:

[0090] User input, where users enter data or perform operations through the platform, such as setting project parameters;

[0091] Send a request. The front end generates the user's operation as an https request. The request data is encrypted using the https protocol before sending to protect the customer's privacy and data security.

[0092] Server processing, the request is sent to the backend server. After receiving the request, the server parses the request data and parameters;

[0093] Database access: the backend generates corresponding database query or operation requests based on the request content and business logic, such as executing database queries or calling other services;

[0094] Return the response. After the backend processes the request, it returns the result to the UI in JSON format.

[0095] Display results: After receiving the response from the server, the UI updates the page display, such as displaying query results or feedback information;

[0096] Real-time update, using the WebSocket protocol and JSON format for data packaging, so that the UI can be updated in real time without reloading the page;

[0097] Error handling: The UI needs to be able to handle and display errors that may occur, such as network problems or server errors.

[0098] In some specific embodiments, the data interaction between the device and the backend database is as follows: Fig. 9 As shown, the steps include:

[0099] Collect data: the device collects image data through cameras and other devices;

[0100] Process data, process and analyze the collected images, and extract the required information.

[0101] Prepare data and package the processed data in JSON format;

[0102] Establish connection: The device establishes a secure connection with the backend database through https;

[0103] Send data, through the established connection, send the data to the backend database;

[0104] Receive data, the backend database receives and stores the data in the corresponding database table;

[0105] Confirm the upload, and the backend database sends a confirmation message to the device, indicating that the data has been uploaded successfully;

[0106] Handle errors. If the upload fails, the device will retry or perform other operations based on the error information.

[0107] Record logs: The backend database records the data interaction process to facilitate subsequent monitoring and troubleshooting.

[0108] In some specific embodiments, the main control unit includes a lower camera position, which is used to control camera movement, image acquisition, and collection of environmental features. The data interaction between the main control unit and the lower camera position is as follows: Fig.10 As shown, the steps include:

[0109] Receive instructions: the master controller receives instructions from the platform;

[0110] Generate instructions. The main control generates motor control instructions and temperature and humidity acquisition instructions according to image acquisition tasks and system requirements;

[0111] Execute tasks. The lower computer drives the motor to perform corresponding actions according to the instructions, adjusts the position and focus of the camera, and completes motor control tasks, camera acquisition tasks, temperature and humidity acquisition tasks, etc. Receive data. The main control receives image data from the camera and processes it. It receives data from the temperature and humidity acquisition module transmitted by the sensor and uses it for system monitoring or adjustment.

[0112] Data processing and interaction: The master controller inputs the collected image data into the AI ​​algorithm for analysis to obtain data such as cell number and fusion degree, and uploads processing results and status information through data interaction with the platform. Heartbeat packet task: The master controller periodically sends heartbeat packets to the platform to maintain online status confirmation, which helps the platform detect whether the master controller is online.

[0113] Receive feedback: the master controller receives status feedback from the slave computer and confirms the command execution result and device status;

[0114] Error handling: if the lower computer reports an error or abnormality, the master control will handle the error or resend the command;

[0115] Feedback to users: The master controller displays the processing results or system status information on the user interface and feeds back to the users.

[0116] In some specific embodiments, the device-side host adopts a single-chip main control processing unit with high reliability and low power consumption. The camera acquisition module adopts an industrial-grade camera, and the motor control module adopts a stepper motor to ensure the precise movement and positioning of the camera. The communication module selects a stable Wi-Fi module to ensure the stability and real-time performance of data transmission. The background database uses an SQL database to store image data and analysis results, with strong transaction support and rich data types. The data storage is designed to store image data and analysis results separately to ensure clear data structure and efficient data access. The database server is hosted on a cloud service provider, providing data backup and disaster recovery measures to ensure data security.

[0117] Embodiment 3

[0118] This embodiment also provides an electronic device, referring to Fig.11 , comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0119] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0120] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. By way of example and not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 404 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (Static Random-Access Memory, abbreviated as SRAM) or a dynamic random access memory (Dynamic Random Access Memory, abbreviated as DRAM), wherein the DRAM can be a fast page mode dynamic random access memory 404 (Fast Page Mode Dynamic Random Access Memory, abbreviated as FPMDRAM), an extended data output dynamic random access memory (Extended Date Out Dynamic Random Access Memory, abbreviated as EDODRAM), a synchronous dynamic random access memory (Synchronous Dynamic Random-Access Memory, abbreviated as SDRAM), etc.

[0121] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0122] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the methods for remote real-time analysis and monitoring of cell growth in the above embodiments.

[0123] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0124] The transmission device 406 can be used to receive or send data via a network. The specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0125] The input and output device 408 is used to input or output information. In this embodiment, the input information may be a cell image sequence, etc., and the output information may be a predicted cell number and a predicted fusion degree.

[0126] Optionally, in this embodiment, the processor 402 may be configured to perform the following steps through a computer program:

[0127] Acquire at least two cell images of the area to be monitored acquired at a preset acquisition interval;

[0128] Inputting the cell image into the trained instance segmentation module to obtain a first cell feature corresponding to each cell image, fusing the first cell feature with the second cell feature to obtain a cell image feature of the current cell image, wherein the first cell feature includes cell type, cell number and cell fusion degree, and the second cell feature is an environmental feature of the growth environment corresponding to the current cell image;

[0129] The cell image features of multiple cell images are arranged in the order of acquisition time to form a cell feature sequence, and the cell feature sequence is input into the trained prediction module to obtain the predicted cell number and predicted fusion degree of the cells to be monitored.

[0130] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0131] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0132] Embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, at this point, it should be noted that, for example, Fig.11 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or storage blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0133] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0134] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for remote real-time analysis and monitoring of cell growth, characterized in that: The following steps are involved: Acquire at least two cell images of the area to be monitored acquired at a preset acquisition interval; Inputting the cell image into the trained instance segmentation module to obtain a first cell feature corresponding to each cell image, fusing the first cell feature with the second cell feature to obtain a cell image feature of the current cell image, wherein the first cell feature includes cell type, cell number and cell fusion degree, and the second cell feature is an environmental feature of the growth environment corresponding to the current cell image; The cell image features of multiple cell images are arranged in the order of acquisition time to form a cell feature sequence, and the cell feature sequence is input into the trained prediction module to obtain the predicted cell number and predicted fusion degree of the cells to be monitored.

2. A method for remote real-time analysis and monitoring of cell growth according to claim 1, characterized in that: In the step of "obtaining at least two cell images of the cells to be monitored acquired at a preset acquisition interval", the cell images are pre-processed by any one or more of image brightness processing, image offset correction processing, image compression processing and image recognition processing.

3. The method for remote real-time analysis and monitoring of cell growth according to claim 1, characterized in that: The instance segmentation module includes a detection unit, a segmentation unit and a classification unit. The detection unit records the position of each cell in the cell image with a detection frame. The segmentation unit is used to determine whether each pixel point in the detection frame is a cell pixel so as to segment the cell contour. The classification unit is used to identify the category of each cell, wherein dead cells are treated as a separate cell category.

4. A method for remote real-time analysis and monitoring of cell growth according to claim 3, characterized in that: Set the detection threshold, retain the detection frames with confidence greater than the detection threshold, then set the initial segmentation threshold, use the initial segmentation threshold to segment each pixel in each detection frame, regard the pixels greater than the initial segmentation threshold as cell pixels, and the pixels less than or equal to the initial segmentation threshold as non-cell pixels. The mask area composed of all cell pixels in each detection frame, finally, dynamically adjust the segmentation threshold of each detection frame based on the mask area of ​​the detection frame with the highest confidence to obtain the dynamic threshold corresponding to each detection frame, and segment each detection frame based on the corresponding dynamic threshold to obtain the cell contour.

5. The method for remote real-time analysis and monitoring of cell growth according to claim 1, characterized in that: The attention mechanism is used to assign weights to cell categories, cell numbers, cell fusion, and environmental features, and the weighted sum is used to obtain cell image features.

6. The method for remote real-time analysis and monitoring of cell growth according to claim 1, characterized in that: The prediction time span is set, and the cell feature sequence is combined with the prediction time span and input into the prediction module to obtain the predicted cell number and predicted fusion degree within the prediction time span.

7. A remote real-time analysis and monitoring device for cell growth, characterized in that: Including platform side, device side and backend database: The platform end is used to send operation instructions to the device end; The device end includes a data acquisition module, a trained instance segmentation module and a trained prediction module, wherein the data acquisition module is used to obtain at least two cell images of cells to be monitored acquired at a preset acquisition interval, input the cell images into the trained instance segmentation module to obtain a first cell feature corresponding to each cell image, fuse the first cell feature and the second cell feature to obtain a cell image feature of the current cell image, wherein the first cell feature includes a cell category, a cell number and a cell fusion degree, and the second cell feature is an environmental feature of a growth environment corresponding to the current cell image, arrange the cell image features of multiple cell images in order of acquisition time to form a cell feature sequence, and input the cell feature sequence into the trained prediction module to obtain a predicted cell number and a predicted fusion degree of the cells to be monitored; The backend database is used to store all information generated by the platform side and the device side, and is used by the platform side and the device side for calling.

8. The device for remote real-time analysis and monitoring of cell growth according to claim 7, characterized in that: The predicted cell number and predicted fusion degree are called in the background database, and the predicted cell number and predicted fusion degree are generated into images, scatter plots, curve graphs or time-lapse videos and displayed on the platform.

9. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the method for remote real-time analysis and monitoring of cell growth according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, wherein the computer program includes a program code for controlling a process to execute a process, wherein the process includes a method for remote real-time analysis and monitoring of cell growth according to any one of claims 1-7.

Images